2023-10-07T23:07:29.777 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-07T23:07:29.777 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} --archive /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060 --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/orig.config.yaml 2023-10-07T23:07:29.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T23:07:29.914 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} email: yweinste@redhat.com first_in_suite: false job_id: '7418060' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-07-1508-quincy sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-07-1508-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_22:15:20 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-07T23:07:29.915 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa; will attempt to use it 2023-10-07T23:07:29.916 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks 2023-10-07T23:07:29.916 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-07T23:07:29.918 INFO:teuthology.task.internal:Checking packages... 2023-10-07T23:07:29.945 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-07T23:07:29.946 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-07T23:07:29.946 INFO:teuthology.packaging:ref: None 2023-10-07T23:07:29.947 INFO:teuthology.packaging:tag: None 2023-10-07T23:07:29.947 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T23:07:29.947 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:07:29.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-07T23:07:30.197 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320-g08cac018-1focal 2023-10-07T23:07:30.197 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-07T23:07:30.204 INFO:teuthology.task.internal:no buildpackages task found 2023-10-07T23:07:30.204 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-07T23:07:30.251 INFO:teuthology.task.internal:Saving configuration 2023-10-07T23:07:30.267 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-07T23:07:30.284 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:07:30.309 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:01:15.903445', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-07T23:07:30.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:01:15.901698', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-10-07T23:07:30.346 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-07T23:07:30.355 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-07T23:07:30.355 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-07T23:07:30.355 INFO:teuthology.run_tasks:Running task console_log... 2023-10-07T23:07:30.440 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd7e35f0040>, signals=[15]) 2023-10-07T23:07:30.440 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-07T23:07:30.449 INFO:teuthology.task.internal:Opening connections... 2023-10-07T23:07:30.449 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:07:30.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:07:30.499 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:07:30.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:07:30.549 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-07T23:07:30.558 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2023-10-07T23:07:30.582 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2023-10-07T23:07:30.582 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2023-10-07T23:07:30.630 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Ubuntu" 2023-10-07T23:07:30.630 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-07T23:07:30.630 INFO:teuthology.orchestra.run.smithi057.stdout:ID=ubuntu 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE=debian 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="20.04" 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-07T23:07:30.631 INFO:teuthology.orchestra.run.smithi057.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-07T23:07:30.632 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_CODENAME=focal 2023-10-07T23:07:30.632 INFO:teuthology.orchestra.run.smithi057.stdout:UBUNTU_CODENAME=focal 2023-10-07T23:07:30.632 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2023-10-07T23:07:30.654 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-10-07T23:07:30.660 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-10-07T23:07:30.661 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Ubuntu" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:ID=ubuntu 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE=debian 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="20.04" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-07T23:07:30.707 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-07T23:07:30.708 INFO:teuthology.orchestra.run.smithi154.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-07T23:07:30.708 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_CODENAME=focal 2023-10-07T23:07:30.708 INFO:teuthology.orchestra.run.smithi154.stdout:UBUNTU_CODENAME=focal 2023-10-07T23:07:30.708 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-10-07T23:07:30.737 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-07T23:07:30.747 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-07T23:07:30.764 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-07T23:07:30.765 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2023-10-07T23:07:30.767 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2023-10-07T23:07:30.773 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-07T23:07:30.780 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-07T23:07:30.780 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2023-10-07T23:07:30.782 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2023-10-07T23:07:30.810 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-07T23:07:30.863 INFO:teuthology.run_tasks:Running task kernel... 2023-10-07T23:07:30.879 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-07T23:07:30.880 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-07T23:07:30.880 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T23:07:30.880 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-07T23:07:30.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:07:30.886 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-07T23:07:30.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:07:30.887 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-10-07T23:07:30.893 INFO:teuthology.orchestra.run.smithi057.stdout:5.4.0-124-generic 2023-10-07T23:07:30.893 INFO:teuthology.task.kernel:Running kernel on smithi057: 5.4.0-124-generic 2023-10-07T23:07:30.893 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get clean 2023-10-07T23:07:30.894 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-124-generic 2023-10-07T23:07:30.894 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.4.0-124-generic 2023-10-07T23:07:30.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2023-10-07T23:07:30.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2023-10-07T23:07:30.998 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2023-10-07T23:07:31.217 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:07:31.246 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-07T23:07:31.260 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-07T23:07:31.375 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-07T23:07:31.375 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:07:31.375 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-10-07T23:07:31.393 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-10-07T23:07:31.626 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-07T23:07:31.737 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-10-07T23:07:31.740 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-07T23:07:31.794 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-07T23:07:31.800 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-07T23:07:31.801 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-07T23:07:31.830 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-07T23:07:31.836 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-07T23:07:31.836 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-07T23:07:31.841 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-07T23:07:31.841 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-07T23:07:31.860 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-07T23:07:31.869 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-07T23:07:31.873 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-07T23:07:31.881 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-07T23:07:31.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-07T23:07:31.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-07T23:07:31.882 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-07T23:07:31.895 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-07T23:07:31.896 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-07T23:07:31.897 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-07T23:07:31.897 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-07T23:07:31.897 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-07T23:07:31.900 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-07T23:07:31.931 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [660 kB] 2023-10-07T23:07:31.974 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [893 kB] 2023-10-07T23:07:32.170 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-07T23:07:32.257 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,500 kB] 2023-10-07T23:07:32.306 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,888 kB] 2023-10-07T23:07:32.358 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-07T23:07:32.373 INFO:teuthology.orchestra.run.smithi057.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-07T23:07:32.373 INFO:teuthology.orchestra.run.smithi057.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-07T23:07:32.374 INFO:teuthology.orchestra.run.smithi057.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-07T23:07:32.442 INFO:teuthology.orchestra.run.smithi057.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-07T23:07:32.446 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [390 kB] 2023-10-07T23:07:32.448 INFO:teuthology.orchestra.run.smithi057.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-07T23:07:32.448 INFO:teuthology.orchestra.run.smithi057.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-07T23:07:32.460 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-10-07T23:07:32.461 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,267 kB] 2023-10-07T23:07:32.470 INFO:teuthology.orchestra.run.smithi057.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-07T23:07:32.481 INFO:teuthology.orchestra.run.smithi057.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [473 kB] 2023-10-07T23:07:32.494 INFO:teuthology.orchestra.run.smithi057.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-10-07T23:07:32.494 INFO:teuthology.orchestra.run.smithi057.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,383 kB] 2023-10-07T23:07:32.528 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-10-07T23:07:32.529 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [317 kB] 2023-10-07T23:07:32.535 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-10-07T23:07:32.535 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [616 kB] 2023-10-07T23:07:32.547 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [890 kB] 2023-10-07T23:07:32.565 INFO:teuthology.orchestra.run.smithi057.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.6 kB] 2023-10-07T23:07:32.566 INFO:teuthology.orchestra.run.smithi057.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [334 kB] 2023-10-07T23:07:32.571 INFO:teuthology.orchestra.run.smithi057.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-10-07T23:07:32.659 INFO:teuthology.orchestra.run.smithi057.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-07T23:07:32.735 INFO:teuthology.orchestra.run.smithi057.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [749 kB] 2023-10-07T23:07:32.747 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [186 kB] 2023-10-07T23:07:33.074 INFO:teuthology.orchestra.run.smithi057.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-07T23:07:33.080 INFO:teuthology.orchestra.run.smithi057.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-07T23:07:33.082 INFO:teuthology.orchestra.run.smithi057.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-07T23:07:33.090 INFO:teuthology.orchestra.run.smithi057.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-07T23:07:33.092 INFO:teuthology.orchestra.run.smithi057.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-07T23:07:33.166 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-10-07T23:07:33.172 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-10-07T23:07:33.179 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-10-07T23:07:33.182 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-10-07T23:07:33.184 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-10-07T23:07:33.324 INFO:teuthology.orchestra.run.smithi057.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,121 kB] 2023-10-07T23:07:33.433 INFO:teuthology.orchestra.run.smithi057.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [268 kB] 2023-10-07T23:07:33.454 INFO:teuthology.orchestra.run.smithi057.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-10-07T23:07:33.455 INFO:teuthology.orchestra.run.smithi057.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-10-07T23:07:33.456 INFO:teuthology.orchestra.run.smithi057.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.8 kB] 2023-10-07T23:07:33.458 INFO:teuthology.orchestra.run.smithi057.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-10-07T23:07:33.459 INFO:teuthology.orchestra.run.smithi057.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-10-07T23:07:33.459 INFO:teuthology.orchestra.run.smithi057.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-10-07T23:07:33.460 INFO:teuthology.orchestra.run.smithi057.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-10-07T23:07:33.464 INFO:teuthology.orchestra.run.smithi057.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-10-07T23:07:33.629 INFO:teuthology.orchestra.run.smithi057.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-10-07T23:07:33.793 INFO:teuthology.orchestra.run.smithi057.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-10-07T23:07:33.845 INFO:teuthology.orchestra.run.smithi057.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-10-07T23:07:35.645 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 17.6 MB in 3s (6,700 kB/s) 2023-10-07T23:07:36.274 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 17.6 MB in 3s (5,522 kB/s) 2023-10-07T23:07:37.145 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:07:37.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-07T23:07:37.241 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:07:37.443 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-07T23:07:37.444 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-07T23:07:37.628 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-07T23:07:37.628 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-07T23:07:37.628 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-07T23:07:37.628 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-07T23:07:37.629 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-07T23:07:37.629 INFO:teuthology.orchestra.run.smithi154.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-07T23:07:37.629 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2023-10-07T23:07:37.629 INFO:teuthology.orchestra.run.smithi154.stdout: thermald 2023-10-07T23:07:37.688 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-07T23:07:37.688 INFO:teuthology.orchestra.run.smithi154.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-07T23:07:37.688 INFO:teuthology.orchestra.run.smithi154.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-07T23:07:37.689 INFO:teuthology.orchestra.run.smithi154.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-07T23:07:37.690 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2023-10-07T23:07:37.690 INFO:teuthology.orchestra.run.smithi154.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-07T23:07:37.692 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:07:37.718 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-07T23:07:37.742 INFO:teuthology.orchestra.run.smithi154.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-07T23:07:37.743 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 77.0 MB of archives. 2023-10-07T23:07:37.743 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-07T23:07:37.743 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-07T23:07:37.788 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:07:37.992 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2023-10-07T23:07:37.993 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2023-10-07T23:07:38.096 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-07T23:07:38.180 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2023-10-07T23:07:38.180 INFO:teuthology.orchestra.run.smithi057.stdout: linux-generic linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-07T23:07:38.180 INFO:teuthology.orchestra.run.smithi057.stdout: linux-headers-generic linux-image-5.4.0-164-generic 2023-10-07T23:07:38.180 INFO:teuthology.orchestra.run.smithi057.stdout: linux-modules-5.4.0-164-generic linux-modules-extra-5.4.0-164-generic 2023-10-07T23:07:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2023-10-07T23:07:38.181 INFO:teuthology.orchestra.run.smithi057.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-10-07T23:07:38.181 INFO:teuthology.orchestra.run.smithi057.stdout:Recommended packages: 2023-10-07T23:07:38.181 INFO:teuthology.orchestra.run.smithi057.stdout: thermald 2023-10-07T23:07:38.192 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-07T23:07:38.238 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2023-10-07T23:07:38.238 INFO:teuthology.orchestra.run.smithi057.stdout: linux-headers-5.4.0-164 linux-headers-5.4.0-164-generic 2023-10-07T23:07:38.239 INFO:teuthology.orchestra.run.smithi057.stdout: linux-image-5.4.0-164-generic linux-modules-5.4.0-164-generic 2023-10-07T23:07:38.239 INFO:teuthology.orchestra.run.smithi057.stdout: linux-modules-extra-5.4.0-164-generic 2023-10-07T23:07:38.239 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages will be upgraded: 2023-10-07T23:07:38.240 INFO:teuthology.orchestra.run.smithi057.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-07T23:07:38.422 INFO:teuthology.orchestra.run.smithi057.stdout:3 upgraded, 5 newly installed, 0 to remove and 271 not upgraded. 2023-10-07T23:07:38.422 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 77.0 MB of archives. 2023-10-07T23:07:38.422 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 380 MB of additional disk space will be used. 2023-10-07T23:07:38.422 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-164-generic amd64 5.4.0-164.181 [14.9 MB] 2023-10-07T23:07:38.662 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-07T23:07:38.663 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-07T23:07:38.663 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-07T23:07:38.753 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-07T23:07:38.763 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-07T23:07:39.457 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 77.0 MB in 1s (72.2 MB/s) 2023-10-07T23:07:39.571 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-164-generic amd64 5.4.0-164.181 [10.5 MB] 2023-10-07T23:07:39.667 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-07T23:07:39.839 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-164-generic amd64 5.4.0-164.181 [39.2 MB] 2023-10-07T23:07:41.136 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.164.161 [1,896 B] 2023-10-07T23:07:41.136 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.164.161 [2,592 B] 2023-10-07T23:07:41.136 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164 all 5.4.0-164.181 [11.0 MB] 2023-10-07T23:07:41.421 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-07T23:07:41.422 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-164-generic amd64 5.4.0-164.181 [1,360 kB] 2023-10-07T23:07:41.425 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:41.441 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.164.161 [2,468 B] 2023-10-07T23:07:41.587 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:42.123 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 77.0 MB in 3s (24.1 MB/s) 2023-10-07T23:07:42.281 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-modules-5.4.0-164-generic. 2023-10-07T23:07:43.563 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-07T23:07:43.580 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:43.679 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:44.026 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-10-07T23:07:44.031 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../0-linux-modules-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:44.187 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:44.825 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-07T23:07:44.838 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:44.901 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:46.230 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-image-5.4.0-164-generic. 2023-10-07T23:07:46.243 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../1-linux-image-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:46.347 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:47.484 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-164-generic. 2023-10-07T23:07:47.497 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:47.542 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:07:49.898 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:07:50.056 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:07:50.393 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:07:50.535 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:07:50.956 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-07T23:07:50.973 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-07T23:07:51.046 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-07T23:07:52.492 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../3-linux-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:07:52.660 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:07:53.070 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:07:53.213 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-image-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:07:53.559 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-headers-5.4.0-164. 2023-10-07T23:07:53.576 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../5-linux-headers-5.4.0-164_5.4.0-164.181_all.deb ... 2023-10-07T23:07:53.617 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-07T23:07:57.364 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-07T23:07:57.386 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:07:57.439 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:00.206 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:08:00.226 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package linux-headers-5.4.0-164-generic. 2023-10-07T23:08:00.250 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../6-linux-headers-5.4.0-164-generic_5.4.0-164.181_amd64.deb ... 2023-10-07T23:08:00.318 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:00.350 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:08:00.703 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-07T23:08:00.855 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:01.006 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:02.241 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:03.205 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.164.161_amd64.deb ... 2023-10-07T23:08:03.332 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking linux-headers-generic (5.4.0.164.161) over (5.4.0.124.125) ... 2023-10-07T23:08:03.402 INFO:teuthology.orchestra.run.smithi154.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:03.402 INFO:teuthology.orchestra.run.smithi154.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:03.568 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-07T23:08:03.666 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-5.4.0-164 (5.4.0-164.181) ... 2023-10-07T23:08:03.694 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:03.788 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:04.880 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-modules-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:04.929 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-07T23:08:05.055 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-07T23:08:05.199 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:05.277 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-07T23:08:05.277 INFO:teuthology.orchestra.run.smithi154.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:06.123 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:07.314 INFO:teuthology.orchestra.run.smithi057.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:07.314 INFO:teuthology.orchestra.run.smithi057.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:07.475 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-headers-generic (5.4.0.164.161) ... 2023-10-07T23:08:07.601 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-modules-extra-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:08.847 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-image-generic (5.4.0.164.161) ... 2023-10-07T23:08:08.953 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up linux-generic (5.4.0.164.161) ... 2023-10-07T23:08:09.071 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for linux-image-5.4.0-164-generic (5.4.0-164.181) ... 2023-10-07T23:08:09.147 INFO:teuthology.orchestra.run.smithi057.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-07T23:08:09.147 INFO:teuthology.orchestra.run.smithi057.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:27.537 INFO:teuthology.orchestra.run.smithi154.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-07T23:08:27.537 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub' 2023-10-07T23:08:27.557 INFO:teuthology.orchestra.run.smithi154.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-07T23:08:27.578 INFO:teuthology.orchestra.run.smithi154.stdout:Generating grub configuration file ... 2023-10-07T23:08:27.933 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:27.973 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:28.384 INFO:teuthology.orchestra.run.smithi154.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-07T23:08:28.407 INFO:teuthology.orchestra.run.smithi154.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-07T23:08:30.989 INFO:teuthology.orchestra.run.smithi154.stdout:done 2023-10-07T23:08:31.410 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2023-10-07T23:08:31.460 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2023-10-07T23:08:31.460 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.164.161 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2023-10-07T23:08:31.461 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2023-10-07T23:08:31.462 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2023-10-07T23:08:31.462 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2023-10-07T23:08:31.462 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-07T23:08:31.462 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T23:08:31.462 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T23:08:31.462 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-07T23:08:31.462 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T23:08:31.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get clean 2023-10-07T23:08:31.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2023-10-07T23:08:31.694 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:08:31.695 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-07T23:08:31.703 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-07T23:08:31.723 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-07T23:08:32.161 INFO:teuthology.orchestra.run.smithi057.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-07T23:08:32.161 INFO:teuthology.orchestra.run.smithi057.stdout:Sourcing file `/etc/default/grub' 2023-10-07T23:08:32.171 INFO:teuthology.orchestra.run.smithi057.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-07T23:08:32.190 INFO:teuthology.orchestra.run.smithi057.stdout:Generating grub configuration file ... 2023-10-07T23:08:32.535 INFO:teuthology.orchestra.run.smithi057.stdout:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:32.559 INFO:teuthology.orchestra.run.smithi057.stdout:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:32.939 INFO:teuthology.orchestra.run.smithi057.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-07T23:08:32.951 INFO:teuthology.orchestra.run.smithi057.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-07T23:08:33.438 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:08:33.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-07T23:08:33.536 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:08:33.736 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-07T23:08:33.736 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-07T23:08:33.958 INFO:teuthology.orchestra.run.smithi154.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-07T23:08:33.958 INFO:teuthology.orchestra.run.smithi154.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-07T23:08:33.960 DEBUG:teuthology.orchestra.run.smithi154:> dpkg -s linux-image-generic 2023-10-07T23:08:33.977 INFO:teuthology.orchestra.run.smithi154.stdout:Package: linux-image-generic 2023-10-07T23:08:33.977 INFO:teuthology.orchestra.run.smithi154.stdout:Status: install ok installed 2023-10-07T23:08:33.977 INFO:teuthology.orchestra.run.smithi154.stdout:Priority: optional 2023-10-07T23:08:33.977 INFO:teuthology.orchestra.run.smithi154.stdout:Section: kernel 2023-10-07T23:08:33.977 INFO:teuthology.orchestra.run.smithi154.stdout:Installed-Size: 20 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Maintainer: Ubuntu Kernel Team 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: amd64 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Source: linux-meta 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Version: 5.4.0.164.161 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Recommends: thermald 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout:Description: Generic Linux kernel image 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout: This package will always depend on the latest generic kernel image 2023-10-07T23:08:33.978 INFO:teuthology.orchestra.run.smithi154.stdout: available. 2023-10-07T23:08:33.979 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-07T23:08:33.985 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.oFfwkuPEk9 2023-10-07T23:08:33.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub/grub.cfg /tmp/tmp.oFfwkuPEk9 2023-10-07T23:08:33.996 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.oFfwkuPEk9 2023-10-07T23:08:34.075 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.oFfwkuPEk9 is 9KB 2023-10-07T23:08:34.089 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.oFfwkuPEk9 2023-10-07T23:08:34.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-07T23:08:34.154 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:08:34.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-07T23:08:34.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-07T23:08:34.194 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-07T23:08:34.194 DEBUG:teuthology.orchestra.run.smithi154:> sudo update-grub 2023-10-07T23:08:35.578 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub' 2023-10-07T23:08:35.588 INFO:teuthology.orchestra.run.smithi154.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-07T23:08:35.601 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2023-10-07T23:08:35.741 INFO:teuthology.orchestra.run.smithi057.stdout:done 2023-10-07T23:08:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:36.184 DEBUG:teuthology.orchestra.run.smithi057:> dpkg -s linux-image-generic 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Package: linux-image-generic 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Status: install ok installed 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Priority: optional 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Section: kernel 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Installed-Size: 20 2023-10-07T23:08:36.225 INFO:teuthology.orchestra.run.smithi057.stdout:Maintainer: Ubuntu Kernel Team 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: amd64 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Source: linux-meta 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Version: 5.4.0.164.161 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-07T23:08:36.226 INFO:teuthology.orchestra.run.smithi057.stdout:Recommends: thermald 2023-10-07T23:08:36.227 INFO:teuthology.orchestra.run.smithi057.stdout:Description: Generic Linux kernel image 2023-10-07T23:08:36.227 INFO:teuthology.orchestra.run.smithi057.stdout: This package will always depend on the latest generic kernel image 2023-10-07T23:08:36.227 INFO:teuthology.orchestra.run.smithi057.stdout: available. 2023-10-07T23:08:36.227 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-164-generic 2023-10-07T23:08:36.227 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-07T23:08:36.228 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-07T23:08:36.228 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-07T23:08:36.228 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=None, version=distro) 2023-10-07T23:08:36.228 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get clean 2023-10-07T23:08:36.285 INFO:teuthology.orchestra.run.smithi154.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-07T23:08:36.297 INFO:teuthology.orchestra.run.smithi154.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-07T23:08:36.366 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2023-10-07T23:08:36.626 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:08:36.627 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-07T23:08:36.698 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-07T23:08:36.788 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-07T23:08:37.774 INFO:teuthology.orchestra.run.smithi154.stderr:done 2023-10-07T23:08:37.776 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2023-10-07T23:08:38.444 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:08:38.470 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-07T23:08:38.538 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:08:38.744 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2023-10-07T23:08:38.744 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2023-10-07T23:08:38.977 INFO:teuthology.orchestra.run.smithi057.stdout:linux-image-generic is already the newest version (5.4.0.164.161). 2023-10-07T23:08:38.977 INFO:teuthology.orchestra.run.smithi057.stdout:0 upgraded, 0 newly installed, 0 to remove and 271 not upgraded. 2023-10-07T23:08:38.979 DEBUG:teuthology.orchestra.run.smithi057:> dpkg -s linux-image-generic 2023-10-07T23:08:39.000 INFO:teuthology.orchestra.run.smithi057.stdout:Package: linux-image-generic 2023-10-07T23:08:39.000 INFO:teuthology.orchestra.run.smithi057.stdout:Status: install ok installed 2023-10-07T23:08:39.000 INFO:teuthology.orchestra.run.smithi057.stdout:Priority: optional 2023-10-07T23:08:39.000 INFO:teuthology.orchestra.run.smithi057.stdout:Section: kernel 2023-10-07T23:08:39.000 INFO:teuthology.orchestra.run.smithi057.stdout:Installed-Size: 20 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Maintainer: Ubuntu Kernel Team 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: amd64 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Source: linux-meta 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Version: 5.4.0.164.161 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Depends: linux-image-5.4.0-164-generic, linux-modules-extra-5.4.0-164-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Recommends: thermald 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout:Description: Generic Linux kernel image 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout: This package will always depend on the latest generic kernel image 2023-10-07T23:08:39.001 INFO:teuthology.orchestra.run.smithi057.stdout: available. 2023-10-07T23:08:39.002 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-10-07T23:08:39.007 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.zIczx4gVFE 2023-10-07T23:08:39.007 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub/grub.cfg /tmp/tmp.zIczx4gVFE 2023-10-07T23:08:39.064 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.zIczx4gVFE 2023-10-07T23:08:39.144 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.zIczx4gVFE is 9KB 2023-10-07T23:08:39.157 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.zIczx4gVFE 2023-10-07T23:08:39.163 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-07T23:08:39.221 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:08:39.221 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-07T23:08:39.222 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-07T23:08:39.258 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-164-generic 2023-10-07T23:08:39.258 DEBUG:teuthology.orchestra.run.smithi057:> sudo update-grub 2023-10-07T23:08:40.587 INFO:teuthology.orchestra.run.smithi057.stderr:Sourcing file `/etc/default/grub' 2023-10-07T23:08:40.595 INFO:teuthology.orchestra.run.smithi057.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-07T23:08:40.610 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2023-10-07T23:08:40.881 INFO:teuthology.orchestra.run.smithi057.stderr:Found linux image: /boot/vmlinuz-5.4.0-164-generic 2023-10-07T23:08:40.892 INFO:teuthology.orchestra.run.smithi057.stderr:Found initrd image: /boot/initrd.img-5.4.0-164-generic 2023-10-07T23:08:41.257 INFO:teuthology.orchestra.run.smithi057.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-10-07T23:08:41.269 INFO:teuthology.orchestra.run.smithi057.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-10-07T23:08:42.964 INFO:teuthology.orchestra.run.smithi057.stderr:done 2023-10-07T23:08:42.966 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2023-10-07T23:09:07.802 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-07T23:09:07.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:09:07.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:09:12.970 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-07T23:09:12.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:09:12.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:09:26.266 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-07T23:09:31.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-10-07T23:09:35.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:09:35.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:09:38.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-07T23:09:40.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:09:40.393 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:09:43.450 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2023-10-07T23:09:50.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:09:50.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:09:55.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:09:55.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:10:22.075 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-10-07T23:10:37.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:10:37.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:10:37.422 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-10-07T23:10:38.021 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:10:38.021 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-164-generic"... 2023-10-07T23:10:38.021 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-10-07T23:10:38.028 INFO:teuthology.orchestra.run.smithi154.stdout:5.4.0-164-generic 2023-10-07T23:10:38.028 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-07T23:10:38.028 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T23:10:38.028 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-07T23:10:39.029 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-07T23:10:39.029 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-07T23:10:39.122 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2023-10-07T23:10:39.146 DEBUG:teuthology.parallel:result is None 2023-10-07T23:10:55.457 DEBUG:teuthology.orchestra.remote:timed out 2023-10-07T23:11:10.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:11:10.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:11:10.669 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-07T23:11:11.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:11:11.291 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-164-generic"... 2023-10-07T23:11:11.292 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2023-10-07T23:11:11.299 INFO:teuthology.orchestra.run.smithi057.stdout:5.4.0-164-generic 2023-10-07T23:11:11.299 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-164-generic vs 5.4.0-164-generic 2023-10-07T23:11:11.299 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-07T23:11:11.299 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-07T23:11:12.300 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-07T23:11:12.301 DEBUG:teuthology.orchestra.run.smithi057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-07T23:11:12.406 INFO:teuthology.orchestra.run.smithi057.stdout:ttyS1 2023-10-07T23:11:12.425 DEBUG:teuthology.parallel:result is None 2023-10-07T23:11:12.426 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-07T23:11:12.433 INFO:teuthology.task.internal:Creating test directory... 2023-10-07T23:11:12.433 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T23:11:12.436 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-07T23:11:12.443 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-07T23:11:12.504 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-07T23:11:12.556 INFO:teuthology.task.internal:Creating archive directory... 2023-10-07T23:11:12.557 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T23:11:12.559 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-07T23:11:12.589 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-07T23:11:12.604 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-07T23:11:12.604 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T23:11:12.629 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-07T23:11:12.645 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:11:12.654 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:11:12.659 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:11:12.670 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-07T23:11:12.672 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-07T23:11:12.683 INFO:teuthology.task.internal:Configuring sudo... 2023-10-07T23:11:12.684 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T23:11:12.686 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-07T23:11:12.732 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-07T23:11:12.743 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-07T23:11:12.743 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T23:11:12.745 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-07T23:11:12.751 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T23:11:12.798 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T23:11:12.804 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:11:12.804 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T23:11:12.825 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-07T23:11:12.832 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-07T23:11:12.839 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:11:12.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-07T23:11:12.864 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2023-10-07T23:11:12.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2023-10-07T23:11:13.000 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-07T23:11:13.028 INFO:teuthology.task.internal:Starting timer... 2023-10-07T23:11:13.028 INFO:teuthology.run_tasks:Running task pcp... 2023-10-07T23:11:13.075 INFO:teuthology.run_tasks:Running task selinux... 2023-10-07T23:11:13.156 DEBUG:teuthology.task.selinux:Excluding smithi057: OS 'ubuntu' does not support SELinux 2023-10-07T23:11:13.156 DEBUG:teuthology.task.selinux:Excluding smithi154: OS 'ubuntu' does not support SELinux 2023-10-07T23:11:13.156 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-07T23:11:13.156 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-07T23:11:13.156 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-07T23:11:13.156 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-07T23:11:13.182 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-07T23:11:13.182 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-07T23:11:13.199 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-07T23:11:13.200 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi154.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-07T23:17:01.449 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com')] 2023-10-07T23:17:01.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:17:01.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:17:01.500 DEBUG:teuthology.orchestra.run.smithi057:> true 2023-10-07T23:17:01.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2023-10-07T23:17:01.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:17:01.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-07T23:17:01.672 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-10-07T23:17:01.796 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-10-07T23:17:01.798 INFO:teuthology.run_tasks:Running task clock... 2023-10-07T23:17:01.811 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-07T23:17:01.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T23:17:01.812 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:17:01.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-07T23:17:01.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:17:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-07T23:17:01.845 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Command line: ntpd -gq 2023-10-07T23:17:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: proto: precision = 0.078 usec (-24) 2023-10-07T23:17:01.846 INFO:teuthology.orchestra.run.smithi057.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-07T23:17:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-07T23:17:01.846 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: restrict ::: KOD does nothing without LIMITED. 2023-10-07T23:17:01.847 INFO:teuthology.orchestra.run.smithi057.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-07T23:17:01.847 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen and drop on 0 v6wildcard [::]:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen normally on 2 lo 127.0.0.1:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen normally on 3 ens1f0 172.21.15.57:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen normally on 4 lo [::1]:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1562%4]:123 2023-10-07T23:17:01.848 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:01 ntpd[22802]: Listening on routing socket on fd #22 for interface updates 2023-10-07T23:17:01.849 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-10-07T23:17:01.849 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Command line: ntpd -gq 2023-10-07T23:17:01.850 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: proto: precision = 0.085 usec (-23) 2023-10-07T23:17:01.850 INFO:teuthology.orchestra.run.smithi154.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-07T23:17:01.851 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-07T23:17:01.851 INFO:teuthology.orchestra.run.smithi154.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-07T23:17:01.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: restrict ::: KOD does nothing without LIMITED. 2023-10-07T23:17:01.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen and drop on 0 v6wildcard [::]:123 2023-10-07T23:17:01.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-07T23:17:01.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen normally on 2 lo 127.0.0.1:123 2023-10-07T23:17:01.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen normally on 3 enp3s0f1 172.21.15.154:123 2023-10-07T23:17:01.853 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen normally on 4 lo [::1]:123 2023-10-07T23:17:01.853 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7761%5]:123 2023-10-07T23:17:01.853 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:01 ntpd[22890]: Listening on routing socket on fd #22 for interface updates 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:02.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:02 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:02.849 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:02 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:02.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:02 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:03.845 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:03 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:03.845 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:03 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:03.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:03 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:04.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:04.848 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:04 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:04.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:04 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:05.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:05 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:05.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:05 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:06.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:06.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:06 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:06.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:06 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:07.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:07 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:07.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:07.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:07 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:08.846 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:08.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:08.847 INFO:teuthology.orchestra.run.smithi057.stdout: 7 Oct 23:17:08 ntpd[22802]: ntpd: time slew +0.001520 s 2023-10-07T23:17:08.847 INFO:teuthology.orchestra.run.smithi057.stdout:ntpd: time slew +0.001520s 2023-10-07T23:17:08.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-07T23:17:08.847 INFO:teuthology.orchestra.run.smithi057.stderr: 7 Oct 23:17:08 ntpd[22802]: can't open /var/log/ntpstats/loopstats.20231007: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:08.850 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:08 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout: remote refid st t when poll reach delay offset jitter 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================== 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:08.907 INFO:teuthology.orchestra.run.smithi057.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:09.851 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/rawstats.20231007: Permission denied 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/peerstats.20231007: Permission denied 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stdout: 7 Oct 23:17:09 ntpd[22890]: ntpd: time slew -0.001000 s 2023-10-07T23:17:09.852 INFO:teuthology.orchestra.run.smithi154.stdout:ntpd: time slew -0.001000s 2023-10-07T23:17:09.853 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-07T23:17:09.853 INFO:teuthology.orchestra.run.smithi154.stderr: 7 Oct 23:17:09 ntpd[22890]: can't open /var/log/ntpstats/loopstats.20231007: Permission denied 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-10-07T23:17:09.919 INFO:teuthology.run_tasks:Running task install... 2023-10-07T23:17:09.928 DEBUG:teuthology.task.install:project ceph 2023-10-07T23:17:09.928 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-07T23:17:09.928 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-07T23:17:09.929 INFO:teuthology.task.install:Using flavor: default 2023-10-07T23:17:09.934 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-07T23:17:09.934 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-07T23:17:09.934 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-key list | grep Ceph 2023-10-07T23:17:09.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-key list | grep Ceph 2023-10-07T23:17:10.031 INFO:teuthology.orchestra.run.smithi154.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-07T23:17:10.032 INFO:teuthology.orchestra.run.smithi057.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-10-07T23:17:10.093 INFO:teuthology.orchestra.run.smithi057.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-07T23:17:10.093 INFO:teuthology.orchestra.run.smithi057.stdout:uid [ unknown] Ceph.com (release key) 2023-10-07T23:17:10.094 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-07T23:17:10.094 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-07T23:17:10.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:17:10.099 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-07T23:17:10.099 INFO:teuthology.orchestra.run.smithi154.stdout:uid [ unknown] Ceph.com (release key) 2023-10-07T23:17:10.099 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-07T23:17:10.099 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2023-10-07T23:17:10.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:17:10.231 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default/ 2023-10-07T23:17:10.231 INFO:teuthology.task.install.deb:Package version is 17.2.6-1320-g08cac018-1focal 2023-10-07T23:17:10.310 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default/ 2023-10-07T23:17:10.311 INFO:teuthology.task.install.deb:Package version is 17.2.6-1320-g08cac018-1focal 2023-10-07T23:17:10.352 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:17:10.352 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-07T23:17:10.368 DEBUG:teuthology.orchestra.run.smithi057:> sudo apt-get update 2023-10-07T23:17:10.427 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:17:10.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-07T23:17:10.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo apt-get update 2023-10-07T23:17:10.559 INFO:teuthology.orchestra.run.smithi057.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal InRelease 2023-10-07T23:17:10.595 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-07T23:17:10.618 INFO:teuthology.orchestra.run.smithi057.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release.gpg 2023-10-07T23:17:10.635 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:17:10.641 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-07T23:17:10.642 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal InRelease 2023-10-07T23:17:10.659 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:6 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-07T23:17:10.663 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-10-07T23:17:10.682 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main i386 Packages [3,366 B] 2023-10-07T23:17:10.687 INFO:teuthology.orchestra.run.smithi154.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal Release.gpg 2023-10-07T23:17:10.705 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-07T23:17:10.711 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-10-07T23:17:10.711 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main i386 Packages [3,366 B] 2023-10-07T23:17:10.748 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-10-07T23:17:10.791 INFO:teuthology.orchestra.run.smithi057.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-07T23:17:11.320 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 28.4 kB in 1s (36.1 kB/s) 2023-10-07T23:17:12.392 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:17:12.419 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1320-g08cac018-1focal cephadm=17.2.6-1320-g08cac018-1focal ceph-mds=17.2.6-1320-g08cac018-1focal ceph-mgr=17.2.6-1320-g08cac018-1focal ceph-common=17.2.6-1320-g08cac018-1focal ceph-fuse=17.2.6-1320-g08cac018-1focal ceph-test=17.2.6-1320-g08cac018-1focal ceph-volume=17.2.6-1320-g08cac018-1focal radosgw=17.2.6-1320-g08cac018-1focal python3-rados=17.2.6-1320-g08cac018-1focal python3-rgw=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal python3-rbd=17.2.6-1320-g08cac018-1focal libcephfs2=17.2.6-1320-g08cac018-1focal libcephfs-dev=17.2.6-1320-g08cac018-1focal librados2=17.2.6-1320-g08cac018-1focal librbd1=17.2.6-1320-g08cac018-1focal rbd-fuse=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal cephfs-shell=17.2.6-1320-g08cac018-1focal cephfs-top=17.2.6-1320-g08cac018-1focal cephfs-mirror=17.2.6-1320-g08cac018-1focal 2023-10-07T23:17:12.488 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:17:12.693 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2023-10-07T23:17:12.694 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2023-10-07T23:17:12.884 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-07T23:17:12.886 INFO:teuthology.orchestra.run.smithi057.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-07T23:17:12.887 INFO:teuthology.orchestra.run.smithi057.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-07T23:17:12.888 INFO:teuthology.orchestra.run.smithi057.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-07T23:17:12.889 INFO:teuthology.orchestra.run.smithi057.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-07T23:17:12.890 INFO:teuthology.orchestra.run.smithi057.stdout:Recommended packages: 2023-10-07T23:17:12.890 INFO:teuthology.orchestra.run.smithi057.stdout: btrfs-tools 2023-10-07T23:17:13.014 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2023-10-07T23:17:13.014 INFO:teuthology.orchestra.run.smithi057.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-07T23:17:13.014 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-07T23:17:13.015 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-07T23:17:13.015 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-07T23:17:13.015 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-07T23:17:13.015 INFO:teuthology.orchestra.run.smithi057.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-07T23:17:13.015 INFO:teuthology.orchestra.run.smithi057.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-07T23:17:13.016 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-07T23:17:13.017 INFO:teuthology.orchestra.run.smithi057.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-07T23:17:13.018 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages will be upgraded: 2023-10-07T23:17:13.018 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 librbd1 2023-10-07T23:17:13.209 INFO:teuthology.orchestra.run.smithi057.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-07T23:17:13.210 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 210 MB of archives. 2023-10-07T23:17:13.210 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-07T23:17:13.210 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-07T23:17:13.293 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-07T23:17:13.293 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1320-g08cac018-1focal [3,501 kB] 2023-10-07T23:17:13.327 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-07T23:17:13.450 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-07T23:17:13.576 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1320-g08cac018-1focal [3,552 kB] 2023-10-07T23:17:13.587 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-07T23:17:13.597 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-07T23:17:13.642 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-07T23:17:13.653 INFO:teuthology.orchestra.run.smithi057.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-07T23:17:13.684 INFO:teuthology.orchestra.run.smithi057.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1320-g08cac018-1focal [760 kB] 2023-10-07T23:17:13.709 INFO:teuthology.orchestra.run.smithi057.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1320-g08cac018-1focal [414 kB] 2023-10-07T23:17:13.722 INFO:teuthology.orchestra.run.smithi057.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1320-g08cac018-1focal [22.6 kB] 2023-10-07T23:17:13.723 INFO:teuthology.orchestra.run.smithi057.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1320-g08cac018-1focal [200 kB] 2023-10-07T23:17:13.735 INFO:teuthology.orchestra.run.smithi057.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-07T23:17:13.736 INFO:teuthology.orchestra.run.smithi057.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1320-g08cac018-1focal [47.2 kB] 2023-10-07T23:17:13.740 INFO:teuthology.orchestra.run.smithi057.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-07T23:17:13.743 INFO:teuthology.orchestra.run.smithi057.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1320-g08cac018-1focal [386 kB] 2023-10-07T23:17:13.749 INFO:teuthology.orchestra.run.smithi057.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1320-g08cac018-1focal [5,568 kB] 2023-10-07T23:17:13.763 INFO:teuthology.orchestra.run.smithi057.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-07T23:17:13.988 INFO:teuthology.orchestra.run.smithi057.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1320-g08cac018-1focal [116 kB] 2023-10-07T23:17:13.990 INFO:teuthology.orchestra.run.smithi057.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1320-g08cac018-1focal [479 kB] 2023-10-07T23:17:14.052 INFO:teuthology.orchestra.run.smithi057.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-07T23:17:14.108 INFO:teuthology.orchestra.run.smithi057.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1320-g08cac018-1focal [27.8 MB] 2023-10-07T23:17:14.486 INFO:teuthology.orchestra.run.smithi057.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-07T23:17:14.668 INFO:teuthology.orchestra.run.smithi057.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-07T23:17:14.699 INFO:teuthology.orchestra.run.smithi057.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-07T23:17:14.705 INFO:teuthology.orchestra.run.smithi057.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-07T23:17:14.730 INFO:teuthology.orchestra.run.smithi057.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-07T23:17:14.735 INFO:teuthology.orchestra.run.smithi057.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-07T23:17:14.737 INFO:teuthology.orchestra.run.smithi057.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-07T23:17:14.750 INFO:teuthology.orchestra.run.smithi057.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-07T23:17:14.752 INFO:teuthology.orchestra.run.smithi057.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-07T23:17:14.754 INFO:teuthology.orchestra.run.smithi057.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-07T23:17:14.919 INFO:teuthology.orchestra.run.smithi057.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-07T23:17:15.083 INFO:teuthology.orchestra.run.smithi057.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-07T23:17:15.224 INFO:teuthology.orchestra.run.smithi057.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-07T23:17:15.243 INFO:teuthology.orchestra.run.smithi057.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-07T23:17:15.308 INFO:teuthology.orchestra.run.smithi057.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-07T23:17:15.375 INFO:teuthology.orchestra.run.smithi057.stdout:Get:40 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1320-g08cac018-1focal [6,450 kB] 2023-10-07T23:17:15.375 INFO:teuthology.orchestra.run.smithi057.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-07T23:17:15.376 INFO:teuthology.orchestra.run.smithi057.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-07T23:17:15.394 INFO:teuthology.orchestra.run.smithi057.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-07T23:17:15.423 INFO:teuthology.orchestra.run.smithi057.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-07T23:17:15.428 INFO:teuthology.orchestra.run.smithi057.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-07T23:17:15.467 INFO:teuthology.orchestra.run.smithi057.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1320-g08cac018-1focal [221 kB] 2023-10-07T23:17:15.473 INFO:teuthology.orchestra.run.smithi057.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1320-g08cac018-1focal [152 kB] 2023-10-07T23:17:15.478 INFO:teuthology.orchestra.run.smithi057.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1320-g08cac018-1focal [1,590 kB] 2023-10-07T23:17:15.522 INFO:teuthology.orchestra.run.smithi057.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-07T23:17:15.530 INFO:teuthology.orchestra.run.smithi057.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1320-g08cac018-1focal [7,846 kB] 2023-10-07T23:17:15.687 INFO:teuthology.orchestra.run.smithi057.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-07T23:17:15.789 INFO:teuthology.orchestra.run.smithi057.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1320-g08cac018-1focal [28.7 MB] 2023-10-07T23:17:15.872 INFO:teuthology.orchestra.run.smithi057.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-07T23:17:15.921 INFO:teuthology.orchestra.run.smithi057.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-07T23:17:16.053 INFO:teuthology.orchestra.run.smithi057.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-07T23:17:16.157 INFO:teuthology.orchestra.run.smithi057.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-07T23:17:16.186 INFO:teuthology.orchestra.run.smithi057.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-07T23:17:16.189 INFO:teuthology.orchestra.run.smithi057.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-07T23:17:16.214 INFO:teuthology.orchestra.run.smithi057.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-07T23:17:16.235 INFO:teuthology.orchestra.run.smithi057.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-07T23:17:16.251 INFO:teuthology.orchestra.run.smithi057.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-07T23:17:16.253 INFO:teuthology.orchestra.run.smithi057.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-07T23:17:16.422 INFO:teuthology.orchestra.run.smithi057.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-07T23:17:16.879 INFO:teuthology.orchestra.run.smithi057.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1320-g08cac018-1focal [3,920 B] 2023-10-07T23:17:16.880 INFO:teuthology.orchestra.run.smithi057.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1320-g08cac018-1focal [869 kB] 2023-10-07T23:17:16.892 INFO:teuthology.orchestra.run.smithi057.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1320-g08cac018-1focal [2,408 kB] 2023-10-07T23:17:16.910 INFO:teuthology.orchestra.run.smithi057.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1320-g08cac018-1focal [79.1 kB] 2023-10-07T23:17:16.911 INFO:teuthology.orchestra.run.smithi057.stdout:Get:68 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1320-g08cac018-1focal [117 kB] 2023-10-07T23:17:16.912 INFO:teuthology.orchestra.run.smithi057.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1320-g08cac018-1focal [1,760 kB] 2023-10-07T23:17:16.946 INFO:teuthology.orchestra.run.smithi057.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1320-g08cac018-1focal [7,875 kB] 2023-10-07T23:17:17.397 INFO:teuthology.orchestra.run.smithi057.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1320-g08cac018-1focal [4,040 B] 2023-10-07T23:17:17.398 INFO:teuthology.orchestra.run.smithi057.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1320-g08cac018-1focal [56.3 MB] 2023-10-07T23:17:17.457 INFO:teuthology.orchestra.run.smithi057.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-07T23:17:17.482 INFO:teuthology.orchestra.run.smithi057.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-07T23:17:17.483 INFO:teuthology.orchestra.run.smithi057.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-07T23:17:17.555 INFO:teuthology.orchestra.run.smithi057.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-07T23:17:17.556 INFO:teuthology.orchestra.run.smithi057.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-07T23:17:17.556 INFO:teuthology.orchestra.run.smithi057.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-07T23:17:17.557 INFO:teuthology.orchestra.run.smithi057.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-07T23:17:17.557 INFO:teuthology.orchestra.run.smithi057.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-07T23:17:17.559 INFO:teuthology.orchestra.run.smithi057.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-07T23:17:17.559 INFO:teuthology.orchestra.run.smithi057.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-07T23:17:17.723 INFO:teuthology.orchestra.run.smithi057.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-07T23:17:17.891 INFO:teuthology.orchestra.run.smithi057.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-07T23:17:17.972 INFO:teuthology.orchestra.run.smithi057.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-07T23:17:18.205 INFO:teuthology.orchestra.run.smithi057.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-07T23:17:18.249 INFO:teuthology.orchestra.run.smithi057.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-07T23:17:18.279 INFO:teuthology.orchestra.run.smithi057.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-07T23:17:18.289 INFO:teuthology.orchestra.run.smithi057.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-07T23:17:18.336 INFO:teuthology.orchestra.run.smithi057.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-07T23:17:18.365 INFO:teuthology.orchestra.run.smithi057.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-07T23:17:18.367 INFO:teuthology.orchestra.run.smithi057.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-07T23:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-07T23:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-07T23:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-07T23:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-07T23:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-07T23:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-07T23:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-07T23:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-07T23:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-07T23:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-07T23:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-07T23:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-07T23:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-07T23:17:20.016 INFO:teuthology.orchestra.run.smithi057.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-07T23:17:20.083 INFO:teuthology.orchestra.run.smithi057.stdout:Get:107 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-07T23:17:20.222 INFO:teuthology.orchestra.run.smithi057.stdout:Get:108 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-07T23:17:20.270 INFO:teuthology.orchestra.run.smithi057.stdout:Get:109 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-07T23:17:20.270 INFO:teuthology.orchestra.run.smithi057.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1320-g08cac018-1focal [229 kB] 2023-10-07T23:17:20.275 INFO:teuthology.orchestra.run.smithi057.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1320-g08cac018-1focal [240 kB] 2023-10-07T23:17:20.280 INFO:teuthology.orchestra.run.smithi057.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1320-g08cac018-1focal [22.0 kB] 2023-10-07T23:17:20.281 INFO:teuthology.orchestra.run.smithi057.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1320-g08cac018-1focal [13.1 kB] 2023-10-07T23:17:20.282 INFO:teuthology.orchestra.run.smithi057.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1320-g08cac018-1focal [21.9 kB] 2023-10-07T23:17:20.282 INFO:teuthology.orchestra.run.smithi057.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1320-g08cac018-1focal [17.0 MB] 2023-10-07T23:17:20.806 INFO:teuthology.orchestra.run.smithi057.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1320-g08cac018-1focal [79.6 kB] 2023-10-07T23:17:21.465 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 210 MB in 8s (27.0 MB/s) 2023-10-07T23:17:21.648 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-07T23:17:21.704 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-07T23:17:21.709 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-07T23:17:21.754 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-07T23:17:22.082 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-07T23:17:22.100 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-07T23:17:22.141 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-07T23:17:22.452 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cmd2. 2023-10-07T23:17:22.470 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-07T23:17:22.510 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-07T23:17:22.855 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-07T23:17:22.875 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-07T23:17:22.955 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-07T23:17:23.319 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../004-librbd1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:23.418 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librbd1 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-07T23:17:24.181 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../005-librados2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:24.306 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librados2 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-07T23:17:24.960 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libcephfs2. 2023-10-07T23:17:24.979 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:25.010 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:25.380 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rados. 2023-10-07T23:17:25.396 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:25.438 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:25.831 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-07T23:17:25.848 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:17:25.884 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:26.113 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cephfs. 2023-10-07T23:17:26.129 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:26.196 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:26.347 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal InRelease 2023-10-07T23:17:26.366 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-10-07T23:17:26.386 INFO:teuthology.orchestra.run.smithi154.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-10-07T23:17:26.526 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-07T23:17:26.544 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:17:26.613 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:26.870 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 28.4 kB in 16s (1,746 B/s) 2023-10-07T23:17:26.933 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-prettytable. 2023-10-07T23:17:26.952 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-07T23:17:27.017 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-07T23:17:27.244 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rbd. 2023-10-07T23:17:27.263 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:27.294 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:27.698 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-07T23:17:27.714 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-07T23:17:27.756 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-07T23:17:27.928 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:17:27.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1320-g08cac018-1focal cephadm=17.2.6-1320-g08cac018-1focal ceph-mds=17.2.6-1320-g08cac018-1focal ceph-mgr=17.2.6-1320-g08cac018-1focal ceph-common=17.2.6-1320-g08cac018-1focal ceph-fuse=17.2.6-1320-g08cac018-1focal ceph-test=17.2.6-1320-g08cac018-1focal ceph-volume=17.2.6-1320-g08cac018-1focal radosgw=17.2.6-1320-g08cac018-1focal python3-rados=17.2.6-1320-g08cac018-1focal python3-rgw=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal python3-rbd=17.2.6-1320-g08cac018-1focal libcephfs2=17.2.6-1320-g08cac018-1focal libcephfs-dev=17.2.6-1320-g08cac018-1focal librados2=17.2.6-1320-g08cac018-1focal librbd1=17.2.6-1320-g08cac018-1focal rbd-fuse=17.2.6-1320-g08cac018-1focal python3-cephfs=17.2.6-1320-g08cac018-1focal cephfs-shell=17.2.6-1320-g08cac018-1focal cephfs-top=17.2.6-1320-g08cac018-1focal cephfs-mirror=17.2.6-1320-g08cac018-1focal 2023-10-07T23:17:28.019 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:17:28.143 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-07T23:17:28.160 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-07T23:17:28.201 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-07T23:17:28.221 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-07T23:17:28.221 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-07T23:17:28.396 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-07T23:17:28.397 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-07T23:17:28.397 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-10-07T23:17:28.398 INFO:teuthology.orchestra.run.smithi154.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-07T23:17:28.400 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-07T23:17:28.401 INFO:teuthology.orchestra.run.smithi154.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout:Recommended packages: 2023-10-07T23:17:28.402 INFO:teuthology.orchestra.run.smithi154.stdout: btrfs-tools 2023-10-07T23:17:28.523 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-07T23:17:28.523 INFO:teuthology.orchestra.run.smithi154.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2023-10-07T23:17:28.524 INFO:teuthology.orchestra.run.smithi154.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2023-10-07T23:17:28.525 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2023-10-07T23:17:28.526 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2023-10-07T23:17:28.527 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 librbd1 2023-10-07T23:17:28.579 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-07T23:17:28.593 INFO:teuthology.orchestra.run.smithi154.stdout:2 upgraded, 114 newly installed, 0 to remove and 253 not upgraded. 2023-10-07T23:17:28.593 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 210 MB of archives. 2023-10-07T23:17:28.593 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 981 MB of additional disk space will be used. 2023-10-07T23:17:28.593 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2023-10-07T23:17:28.596 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-07T23:17:28.614 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-10-07T23:17:28.623 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2023-10-07T23:17:28.638 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-07T23:17:28.652 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-10-07T23:17:28.686 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-10-07T23:17:28.688 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-10-07T23:17:28.700 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-10-07T23:17:28.703 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-10-07T23:17:28.723 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-10-07T23:17:28.724 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-10-07T23:17:28.730 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-10-07T23:17:28.761 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-10-07T23:17:28.763 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-10-07T23:17:28.770 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-10-07T23:17:28.772 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-10-07T23:17:28.773 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-10-07T23:17:28.775 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-10-07T23:17:28.776 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-10-07T23:17:28.776 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-10-07T23:17:28.777 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-10-07T23:17:28.790 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-10-07T23:17:28.790 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-10-07T23:17:28.809 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-10-07T23:17:28.810 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-10-07T23:17:28.811 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-07T23:17:28.812 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-10-07T23:17:28.812 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-10-07T23:17:28.814 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-10-07T23:17:28.814 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-10-07T23:17:28.816 INFO:teuthology.orchestra.run.smithi154.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-10-07T23:17:28.829 INFO:teuthology.orchestra.run.smithi154.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-10-07T23:17:28.830 INFO:teuthology.orchestra.run.smithi154.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-10-07T23:17:28.849 INFO:teuthology.orchestra.run.smithi154.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-10-07T23:17:28.849 INFO:teuthology.orchestra.run.smithi154.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-10-07T23:17:28.852 INFO:teuthology.orchestra.run.smithi154.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-10-07T23:17:28.855 INFO:teuthology.orchestra.run.smithi154.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-10-07T23:17:28.859 INFO:teuthology.orchestra.run.smithi154.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-10-07T23:17:28.861 INFO:teuthology.orchestra.run.smithi154.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1320-g08cac018-1focal [3,501 kB] 2023-10-07T23:17:28.865 INFO:teuthology.orchestra.run.smithi154.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-10-07T23:17:28.868 INFO:teuthology.orchestra.run.smithi154.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-07T23:17:28.871 INFO:teuthology.orchestra.run.smithi154.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-10-07T23:17:28.875 INFO:teuthology.orchestra.run.smithi154.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-10-07T23:17:28.879 INFO:teuthology.orchestra.run.smithi154.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-10-07T23:17:28.888 INFO:teuthology.orchestra.run.smithi154.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-10-07T23:17:28.890 INFO:teuthology.orchestra.run.smithi154.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-10-07T23:17:28.892 INFO:teuthology.orchestra.run.smithi154.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-10-07T23:17:29.001 INFO:teuthology.orchestra.run.smithi154.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-10-07T23:17:29.008 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-07T23:17:29.024 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-07T23:17:29.066 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-07T23:17:29.096 INFO:teuthology.orchestra.run.smithi154.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-10-07T23:17:29.100 INFO:teuthology.orchestra.run.smithi154.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-10-07T23:17:29.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-10-07T23:17:29.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-10-07T23:17:29.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-10-07T23:17:29.117 INFO:teuthology.orchestra.run.smithi154.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-10-07T23:17:29.118 INFO:teuthology.orchestra.run.smithi154.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-10-07T23:17:29.118 INFO:teuthology.orchestra.run.smithi154.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-10-07T23:17:29.118 INFO:teuthology.orchestra.run.smithi154.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-10-07T23:17:29.119 INFO:teuthology.orchestra.run.smithi154.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-10-07T23:17:29.137 INFO:teuthology.orchestra.run.smithi154.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-10-07T23:17:29.138 INFO:teuthology.orchestra.run.smithi154.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-10-07T23:17:29.141 INFO:teuthology.orchestra.run.smithi154.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-10-07T23:17:29.156 INFO:teuthology.orchestra.run.smithi154.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-10-07T23:17:29.158 INFO:teuthology.orchestra.run.smithi154.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-10-07T23:17:29.159 INFO:teuthology.orchestra.run.smithi154.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-10-07T23:17:29.164 INFO:teuthology.orchestra.run.smithi154.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-10-07T23:17:29.168 INFO:teuthology.orchestra.run.smithi154.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2023-10-07T23:17:29.168 INFO:teuthology.orchestra.run.smithi154.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-10-07T23:17:29.185 INFO:teuthology.orchestra.run.smithi154.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-10-07T23:17:29.189 INFO:teuthology.orchestra.run.smithi154.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-10-07T23:17:29.251 INFO:teuthology.orchestra.run.smithi154.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-10-07T23:17:29.252 INFO:teuthology.orchestra.run.smithi154.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-10-07T23:17:29.252 INFO:teuthology.orchestra.run.smithi154.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-10-07T23:17:29.253 INFO:teuthology.orchestra.run.smithi154.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-10-07T23:17:29.253 INFO:teuthology.orchestra.run.smithi154.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-10-07T23:17:29.254 INFO:teuthology.orchestra.run.smithi154.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-10-07T23:17:29.268 INFO:teuthology.orchestra.run.smithi154.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-10-07T23:17:29.269 INFO:teuthology.orchestra.run.smithi154.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-07T23:17:29.295 INFO:teuthology.orchestra.run.smithi154.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-10-07T23:17:29.295 INFO:teuthology.orchestra.run.smithi154.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-10-07T23:17:29.297 INFO:teuthology.orchestra.run.smithi154.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-10-07T23:17:29.299 INFO:teuthology.orchestra.run.smithi154.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-10-07T23:17:29.299 INFO:teuthology.orchestra.run.smithi154.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-10-07T23:17:29.301 INFO:teuthology.orchestra.run.smithi154.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-10-07T23:17:29.302 INFO:teuthology.orchestra.run.smithi154.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-10-07T23:17:29.428 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-07T23:17:29.428 INFO:teuthology.orchestra.run.smithi154.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1320-g08cac018-1focal [3,552 kB] 2023-10-07T23:17:29.445 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-07T23:17:29.486 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-07T23:17:29.696 INFO:teuthology.orchestra.run.smithi154.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1320-g08cac018-1focal [760 kB] 2023-10-07T23:17:29.730 INFO:teuthology.orchestra.run.smithi154.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1320-g08cac018-1focal [414 kB] 2023-10-07T23:17:29.780 INFO:teuthology.orchestra.run.smithi154.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1320-g08cac018-1focal [22.6 kB] 2023-10-07T23:17:29.780 INFO:teuthology.orchestra.run.smithi154.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1320-g08cac018-1focal [200 kB] 2023-10-07T23:17:29.788 INFO:teuthology.orchestra.run.smithi154.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1320-g08cac018-1focal [47.2 kB] 2023-10-07T23:17:29.791 INFO:teuthology.orchestra.run.smithi154.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1320-g08cac018-1focal [386 kB] 2023-10-07T23:17:29.809 INFO:teuthology.orchestra.run.smithi154.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1320-g08cac018-1focal [5,568 kB] 2023-10-07T23:17:29.948 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-07T23:17:29.967 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:17:30.015 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:17:30.268 INFO:teuthology.orchestra.run.smithi154.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1320-g08cac018-1focal [116 kB] 2023-10-07T23:17:30.270 INFO:teuthology.orchestra.run.smithi154.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1320-g08cac018-1focal [479 kB] 2023-10-07T23:17:30.281 INFO:teuthology.orchestra.run.smithi154.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1320-g08cac018-1focal [27.8 MB] 2023-10-07T23:17:30.619 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-07T23:17:30.636 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:17:30.677 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:17:31.072 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-07T23:17:31.088 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:17:31.130 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:17:31.517 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-07T23:17:31.533 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-07T23:17:31.599 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-07T23:17:31.961 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package librgw2. 2023-10-07T23:17:31.977 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../022-librgw2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:32.046 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:32.121 INFO:teuthology.orchestra.run.smithi154.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1320-g08cac018-1focal [6,450 kB] 2023-10-07T23:17:32.496 INFO:teuthology.orchestra.run.smithi154.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1320-g08cac018-1focal [221 kB] 2023-10-07T23:17:32.505 INFO:teuthology.orchestra.run.smithi154.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1320-g08cac018-1focal [152 kB] 2023-10-07T23:17:32.526 INFO:teuthology.orchestra.run.smithi154.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1320-g08cac018-1focal [1,590 kB] 2023-10-07T23:17:32.658 INFO:teuthology.orchestra.run.smithi154.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1320-g08cac018-1focal [7,846 kB] 2023-10-07T23:17:32.807 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rgw. 2023-10-07T23:17:32.826 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:32.882 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:33.227 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liboath0. 2023-10-07T23:17:33.244 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-07T23:17:33.301 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-07T23:17:33.599 INFO:teuthology.orchestra.run.smithi154.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1320-g08cac018-1focal [28.7 MB] 2023-10-07T23:17:33.632 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libradosstriper1. 2023-10-07T23:17:33.651 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:33.715 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:34.310 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-common. 2023-10-07T23:17:34.326 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:34.368 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:37.289 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-base. 2023-10-07T23:17:37.307 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:37.366 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:38.410 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cryptography. 2023-10-07T23:17:38.426 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-07T23:17:38.468 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-07T23:17:38.847 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-openssl. 2023-10-07T23:17:38.866 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-07T23:17:38.905 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-07T23:17:39.175 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-07T23:17:39.191 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:17:39.258 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:39.634 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-07T23:17:39.654 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-07T23:17:39.681 INFO:teuthology.orchestra.run.smithi154.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1320-g08cac018-1focal [3,920 B] 2023-10-07T23:17:39.682 INFO:teuthology.orchestra.run.smithi154.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1320-g08cac018-1focal [869 kB] 2023-10-07T23:17:39.704 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-07T23:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1320-g08cac018-1focal [2,408 kB] 2023-10-07T23:17:40.065 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-07T23:17:40.084 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-07T23:17:40.123 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-07T23:17:40.404 INFO:teuthology.orchestra.run.smithi154.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1320-g08cac018-1focal [79.1 kB] 2023-10-07T23:17:40.407 INFO:teuthology.orchestra.run.smithi154.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1320-g08cac018-1focal [117 kB] 2023-10-07T23:17:40.467 INFO:teuthology.orchestra.run.smithi154.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1320-g08cac018-1focal [1,760 kB] 2023-10-07T23:17:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-natsort. 2023-10-07T23:17:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-07T23:17:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-07T23:17:40.862 INFO:teuthology.orchestra.run.smithi154.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1320-g08cac018-1focal [7,875 kB] 2023-10-07T23:17:40.896 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jwt. 2023-10-07T23:17:40.915 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-07T23:17:40.954 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-07T23:17:41.316 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-logutils. 2023-10-07T23:17:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-07T23:17:41.374 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-07T23:17:41.677 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-07T23:17:41.695 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-07T23:17:41.751 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-07T23:17:42.054 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-mako. 2023-10-07T23:17:42.070 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-07T23:17:42.146 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-07T23:17:42.474 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-07T23:17:42.490 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-07T23:17:42.557 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-07T23:17:42.843 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-07T23:17:42.862 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-07T23:17:42.901 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-07T23:17:42.941 INFO:teuthology.orchestra.run.smithi154.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1320-g08cac018-1focal [4,040 B] 2023-10-07T23:17:42.941 INFO:teuthology.orchestra.run.smithi154.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1320-g08cac018-1focal [56.3 MB] 2023-10-07T23:17:43.229 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-webob. 2023-10-07T23:17:43.245 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-07T23:17:43.287 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-07T23:17:43.623 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-07T23:17:43.643 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-07T23:17:43.681 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-07T23:17:44.018 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-bs4. 2023-10-07T23:17:44.033 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-07T23:17:44.076 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-07T23:17:44.454 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-waitress. 2023-10-07T23:17:44.472 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-07T23:17:44.515 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-07T23:17:44.882 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-tempita. 2023-10-07T23:17:44.902 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-07T23:17:44.940 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-07T23:17:45.259 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-paste. 2023-10-07T23:17:45.279 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-07T23:17:45.317 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-07T23:17:46.387 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-07T23:17:46.407 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-07T23:17:46.470 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-07T23:17:46.782 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-07T23:17:46.800 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-07T23:17:46.864 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-07T23:17:47.151 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-webtest. 2023-10-07T23:17:47.171 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-07T23:17:47.209 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-07T23:17:47.554 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pecan. 2023-10-07T23:17:47.570 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-07T23:17:47.612 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-07T23:17:48.008 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-07T23:17:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-07T23:17:48.066 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-07T23:17:48.444 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-07T23:17:48.461 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:48.502 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:48.873 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr. 2023-10-07T23:17:48.891 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:48.930 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:49.450 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mon. 2023-10-07T23:17:49.470 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:49.508 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:50.828 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-osd. 2023-10-07T23:17:50.845 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:50.886 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:53.764 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph. 2023-10-07T23:17:53.781 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../055-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:53.822 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:53.955 INFO:teuthology.orchestra.run.smithi154.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1320-g08cac018-1focal [229 kB] 2023-10-07T23:17:53.959 INFO:teuthology.orchestra.run.smithi154.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.6-1320-g08cac018-1focal [240 kB] 2023-10-07T23:17:53.964 INFO:teuthology.orchestra.run.smithi154.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.6-1320-g08cac018-1focal [22.0 kB] 2023-10-07T23:17:53.965 INFO:teuthology.orchestra.run.smithi154.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.6-1320-g08cac018-1focal [13.1 kB] 2023-10-07T23:17:53.965 INFO:teuthology.orchestra.run.smithi154.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1320-g08cac018-1focal [21.9 kB] 2023-10-07T23:17:53.966 INFO:teuthology.orchestra.run.smithi154.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1320-g08cac018-1focal [17.0 MB] 2023-10-07T23:17:54.183 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-fuse. 2023-10-07T23:17:54.200 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:54.241 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:54.752 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mds. 2023-10-07T23:17:54.769 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:54.802 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:55.297 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package cephadm. 2023-10-07T23:17:55.313 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../058-cephadm_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:55.380 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:55.683 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-jinja2. 2023-10-07T23:17:55.703 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-07T23:17:55.766 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-07T23:17:56.085 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-07T23:17:56.101 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:17:56.143 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:56.522 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-07T23:17:56.540 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-07T23:17:56.613 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-07T23:17:56.883 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libjs-underscore. 2023-10-07T23:17:56.900 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-07T23:17:56.940 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-07T23:17:57.045 INFO:teuthology.orchestra.run.smithi154.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1320-g08cac018-1focal [79.6 kB] 2023-10-07T23:17:57.235 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-07T23:17:57.250 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-07T23:17:57.293 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-07T23:17:57.645 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-routes. 2023-10-07T23:17:57.661 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-07T23:17:57.679 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 210 MB in 29s (7,361 kB/s) 2023-10-07T23:17:57.695 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-07T23:17:57.829 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-07T23:17:57.883 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141736 files and directories currently installed.) 2023-10-07T23:17:57.887 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2023-10-07T23:17:57.927 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2023-10-07T23:17:58.023 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-07T23:17:58.043 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:17:58.081 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:17:58.221 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-07T23:17:58.241 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-10-07T23:17:58.280 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-07T23:17:58.566 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cmd2. 2023-10-07T23:17:58.581 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2023-10-07T23:17:58.609 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-decorator. 2023-10-07T23:17:58.624 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2023-10-07T23:17:58.628 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-07T23:17:58.692 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-07T23:17:58.885 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-10-07T23:17:58.902 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-10-07T23:17:58.968 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-07T23:17:58.995 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-07T23:17:59.012 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-07T23:17:59.053 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-07T23:17:59.331 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../004-librbd1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:17:59.397 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-scipy. 2023-10-07T23:17:59.413 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-07T23:17:59.455 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-07T23:17:59.456 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librbd1 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-07T23:18:00.764 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-07T23:18:00.781 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-07T23:18:00.847 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:00.918 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../005-librados2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:01.019 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librados2 (17.2.6-1320-g08cac018-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-10-07T23:18:01.401 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-joblib. 2023-10-07T23:18:01.419 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-07T23:18:01.459 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-07T23:18:01.690 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs2. 2023-10-07T23:18:01.707 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../006-libcephfs2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:01.748 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:02.076 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rados. 2023-10-07T23:18:02.092 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../007-python3-rados_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:02.134 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:02.271 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-sklearn. 2023-10-07T23:18:02.287 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-07T23:18:02.328 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:02.504 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-07T23:18:02.519 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:02.562 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:02.802 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cephfs. 2023-10-07T23:18:02.819 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../009-python3-cephfs_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:02.861 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:03.070 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-07T23:18:03.088 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:03.128 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:03.185 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-07T23:18:03.205 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:03.244 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:03.589 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-prettytable. 2023-10-07T23:18:03.609 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2023-10-07T23:18:03.653 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-10-07T23:18:03.906 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rbd. 2023-10-07T23:18:03.922 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../012-python3-rbd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:03.964 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:04.231 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-cachetools. 2023-10-07T23:18:04.251 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-07T23:18:04.300 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-10-07T23:18:04.318 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-10-07T23:18:04.322 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-07T23:18:04.358 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-07T23:18:04.625 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-07T23:18:04.641 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-07T23:18:04.683 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-07T23:18:04.687 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-10-07T23:18:04.705 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2023-10-07T23:18:04.745 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-10-07T23:18:05.044 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-07T23:18:05.061 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-07T23:18:05.090 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-07T23:18:05.102 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-07T23:18:05.109 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-10-07T23:18:05.148 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-07T23:18:05.447 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-rsa. 2023-10-07T23:18:05.467 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-07T23:18:05.505 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-07T23:18:05.526 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-07T23:18:05.543 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-10-07T23:18:05.584 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-07T23:18:05.834 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-google-auth. 2023-10-07T23:18:05.850 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-07T23:18:05.864 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-07T23:18:05.880 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-10-07T23:18:05.891 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-07T23:18:05.914 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-07T23:18:06.219 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-blinker. 2023-10-07T23:18:06.239 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-07T23:18:06.277 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-07T23:18:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-07T23:18:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:18:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:06.613 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-07T23:18:06.629 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-07T23:18:06.671 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-07T23:18:06.986 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-07T23:18:07.003 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:18:07.033 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-07T23:18:07.052 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-07T23:18:07.053 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:07.091 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-07T23:18:07.414 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-07T23:18:07.431 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-10-07T23:18:07.464 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:07.513 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-websocket. 2023-10-07T23:18:07.529 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-07T23:18:07.570 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-07T23:18:07.876 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-10-07T23:18:07.892 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-10-07T23:18:07.916 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-07T23:18:07.933 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-07T23:18:07.934 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-07T23:18:07.974 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-07T23:18:08.296 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package librgw2. 2023-10-07T23:18:08.311 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../022-librgw2_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:08.353 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:08.538 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-07T23:18:08.556 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:08.595 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:08.957 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-07T23:18:08.976 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-07T23:18:09.015 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-07T23:18:09.123 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rgw. 2023-10-07T23:18:09.139 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../023-python3-rgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:09.172 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:09.413 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-07T23:18:09.430 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:18:09.471 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:09.510 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liboath0. 2023-10-07T23:18:09.526 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2023-10-07T23:18:09.568 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-10-07T23:18:09.775 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package jq. 2023-10-07T23:18:09.792 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:18:09.833 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libradosstriper1. 2023-10-07T23:18:09.938 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../025-libradosstriper1_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:09.979 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:10.086 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package socat. 2023-10-07T23:18:10.104 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-07T23:18:10.169 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-07T23:18:10.391 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-common. 2023-10-07T23:18:10.407 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../026-ceph-common_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:10.449 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:10.489 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package xmlstarlet. 2023-10-07T23:18:10.506 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-07T23:18:10.546 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-07T23:18:10.934 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-test. 2023-10-07T23:18:10.954 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:11.017 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:13.105 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-base. 2023-10-07T23:18:13.120 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../027-ceph-base_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:13.173 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:14.134 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cryptography. 2023-10-07T23:18:14.149 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-10-07T23:18:14.192 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-07T23:18:14.579 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-openssl. 2023-10-07T23:18:14.595 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2023-10-07T23:18:14.637 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-10-07T23:18:14.924 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-07T23:18:14.942 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../030-ceph-mgr-modules-core_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:14.982 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:15.371 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-07T23:18:15.387 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../031-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-10-07T23:18:15.429 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-07T23:18:15.732 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-07T23:18:15.748 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../032-python3-cherrypy3_8.9.1-6_all.deb ... 2023-10-07T23:18:15.790 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-10-07T23:18:16.135 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-natsort. 2023-10-07T23:18:16.146 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ceph-volume. 2023-10-07T23:18:16.154 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../033-python3-natsort_7.0.1-1_all.deb ... 2023-10-07T23:18:16.162 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:16.193 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-10-07T23:18:16.204 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:16.513 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jwt. 2023-10-07T23:18:16.531 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../034-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-10-07T23:18:16.571 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-07T23:18:16.626 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-07T23:18:16.644 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:16.719 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:16.875 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-logutils. 2023-10-07T23:18:16.891 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-7_all.deb ... 2023-10-07T23:18:16.933 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-10-07T23:18:17.042 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-colorama. 2023-10-07T23:18:17.059 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-07T23:18:17.126 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-07T23:18:17.236 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-07T23:18:17.254 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../036-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-10-07T23:18:17.317 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-10-07T23:18:17.387 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package cephfs-shell. 2023-10-07T23:18:17.403 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:17.444 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:17.605 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-mako. 2023-10-07T23:18:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../037-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-10-07T23:18:17.665 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-07T23:18:17.721 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package cephfs-top. 2023-10-07T23:18:17.740 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:17.779 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:17.983 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-07T23:18:18.002 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-07T23:18:18.056 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-07T23:18:18.058 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-07T23:18:18.073 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-07T23:18:18.114 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:18.353 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-07T23:18:18.369 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-10-07T23:18:18.411 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-10-07T23:18:18.651 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-07T23:18:18.669 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:18.709 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:18.741 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webob. 2023-10-07T23:18:18.757 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.5-2_all.deb ... 2023-10-07T23:18:18.799 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-10-07T23:18:19.127 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-soupsieve. 2023-10-07T23:18:19.129 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package nvme-cli. 2023-10-07T23:18:19.143 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../041-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-10-07T23:18:19.146 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-07T23:18:19.185 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-07T23:18:19.238 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-07T23:18:19.488 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-bs4. 2023-10-07T23:18:19.506 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../042-python3-bs4_4.8.2-1_all.deb ... 2023-10-07T23:18:19.546 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-10-07T23:18:19.550 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-07T23:18:19.567 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-07T23:18:19.642 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:19.907 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-waitress. 2023-10-07T23:18:19.923 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../043-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-10-07T23:18:19.969 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-07T23:18:20.294 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-tempita. 2023-10-07T23:18:20.311 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6_all.deb ... 2023-10-07T23:18:20.353 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-10-07T23:18:20.663 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-paste. 2023-10-07T23:18:20.679 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../045-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-10-07T23:18:20.721 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-10-07T23:18:21.075 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-07T23:18:21.093 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-10-07T23:18:21.133 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-10-07T23:18:21.367 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-07T23:18:21.387 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-07T23:18:21.426 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-07T23:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-07T23:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.0-1_all.deb ... 2023-10-07T23:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-10-07T23:18:21.795 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-07T23:18:21.805 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webtest. 2023-10-07T23:18:21.815 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-07T23:18:21.821 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../048-python3-webtest_2.0.32-2_all.deb ... 2023-10-07T23:18:21.853 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-07T23:18:21.863 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-10-07T23:18:22.158 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pecan. 2023-10-07T23:18:22.174 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-10-07T23:18:22.181 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-attr. 2023-10-07T23:18:22.201 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-07T23:18:22.216 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-10-07T23:18:22.239 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-07T23:18:22.604 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-07T23:18:22.609 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-webencodings. 2023-10-07T23:18:22.620 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../050-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-10-07T23:18:22.626 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-07T23:18:22.662 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-07T23:18:22.667 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-07T23:18:22.994 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-html5lib. 2023-10-07T23:18:23.011 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-07T23:18:23.032 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-07T23:18:23.050 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../051-libsqlite3-mod-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:23.061 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-07T23:18:23.090 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:23.426 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr. 2023-10-07T23:18:23.431 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-07T23:18:23.446 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../052-ceph-mgr_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:23.447 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-07T23:18:23.484 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:23.488 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-07T23:18:23.963 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mon. 2023-10-07T23:18:23.981 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../053-ceph-mon_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:23.984 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-packaging. 2023-10-07T23:18:24.004 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-07T23:18:24.020 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:24.067 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-07T23:18:24.370 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pastescript. 2023-10-07T23:18:24.387 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-07T23:18:24.461 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-07T23:18:24.781 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pluggy. 2023-10-07T23:18:24.798 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-07T23:18:24.864 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-07T23:18:25.171 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-psutil. 2023-10-07T23:18:25.188 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-07T23:18:25.191 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-osd. 2023-10-07T23:18:25.207 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../054-ceph-osd_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:25.229 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-07T23:18:25.249 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:25.573 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-py. 2023-10-07T23:18:25.592 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-07T23:18:25.623 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-07T23:18:25.934 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-07T23:18:25.954 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-07T23:18:25.992 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-07T23:18:26.353 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-pytest. 2023-10-07T23:18:26.370 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-07T23:18:26.420 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-07T23:18:26.748 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package python3-simplejson. 2023-10-07T23:18:26.766 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-07T23:18:26.798 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-07T23:18:27.084 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-07T23:18:27.101 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-07T23:18:27.142 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-07T23:18:27.927 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph. 2023-10-07T23:18:27.943 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../055-ceph_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:27.972 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package radosgw. 2023-10-07T23:18:27.985 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:27.989 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../114-radosgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:28.033 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:28.304 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-fuse. 2023-10-07T23:18:28.320 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../056-ceph-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:28.362 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:28.766 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mds. 2023-10-07T23:18:28.783 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../057-ceph-mds_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:28.824 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:29.352 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephadm. 2023-10-07T23:18:29.368 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../058-cephadm_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:29.410 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:29.729 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-jinja2. 2023-10-07T23:18:29.745 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2_all.deb ... 2023-10-07T23:18:29.787 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-10-07T23:18:29.802 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package rbd-fuse. 2023-10-07T23:18:29.821 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:29.860 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:30.140 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-07T23:18:30.158 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:30.198 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:30.200 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-07T23:18:30.532 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-07T23:18:30.535 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-07T23:18:30.551 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../061-python3-repoze.lru_0.7-2_all.deb ... 2023-10-07T23:18:30.593 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-07T23:18:30.651 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-07T23:18:30.810 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-07T23:18:30.845 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-underscore. 2023-10-07T23:18:30.861 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../062-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-10-07T23:18:30.904 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-07T23:18:31.020 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:31.215 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-10-07T23:18:31.231 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../063-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-10-07T23:18:31.264 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-07T23:18:31.681 INFO:teuthology.orchestra.run.smithi057.stdout:Adding system user cephadm....done 2023-10-07T23:18:31.834 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:31.960 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-07T23:18:32.026 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-routes. 2023-10-07T23:18:32.043 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../064-python3-routes_2.4.1-2_all.deb ... 2023-10-07T23:18:32.083 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-10-07T23:18:32.354 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-07T23:18:32.445 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-07T23:18:32.461 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../065-ceph-mgr-dashboard_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:32.503 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:33.015 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-decorator. 2023-10-07T23:18:33.030 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:33.031 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../066-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-10-07T23:18:33.073 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-10-07T23:18:33.156 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-07T23:18:33.282 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-07T23:18:33.376 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-10-07T23:18:33.391 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../067-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-10-07T23:18:33.434 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-07T23:18:33.608 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-07T23:18:33.778 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-scipy. 2023-10-07T23:18:33.795 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../068-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-10-07T23:18:33.836 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-10-07T23:18:34.018 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-07T23:18:34.303 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-07T23:18:34.604 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-07T23:18:34.930 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-07T23:18:35.137 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-10-07T23:18:35.156 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../069-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-10-07T23:18:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:35.223 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-07T23:18:35.541 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:35.757 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-joblib. 2023-10-07T23:18:35.777 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../070-python3-joblib_0.14.0-3_all.deb ... 2023-10-07T23:18:35.815 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-10-07T23:18:35.825 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:35.968 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-07T23:18:36.185 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-sklearn. 2023-10-07T23:18:36.201 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../071-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-10-07T23:18:36.243 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:36.260 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-07T23:18:36.571 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-07T23:18:36.857 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-07T23:18:36.901 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-07T23:18:36.919 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../072-ceph-mgr-diskprediction-local_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:36.959 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:36.983 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-07T23:18:37.351 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-07T23:18:37.643 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-07T23:18:37.928 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-07T23:18:38.079 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-cachetools. 2023-10-07T23:18:38.095 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../073-python3-cachetools_4.0.0-1_all.deb ... 2023-10-07T23:18:38.136 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-10-07T23:18:38.287 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-07T23:18:38.431 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-07T23:18:38.447 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../074-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-10-07T23:18:38.489 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-10-07T23:18:38.573 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-07T23:18:38.826 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-07T23:18:38.846 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../075-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-10-07T23:18:38.884 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-07T23:18:38.925 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-07T23:18:39.170 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-rsa. 2023-10-07T23:18:39.187 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../076-python3-rsa_4.0-3_all.deb ... 2023-10-07T23:18:39.210 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-07T23:18:39.228 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-rsa (4.0-3) ... 2023-10-07T23:18:39.548 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-google-auth. 2023-10-07T23:18:39.552 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-07T23:18:39.566 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-2_all.deb ... 2023-10-07T23:18:39.606 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-10-07T23:18:39.712 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-07T23:18:39.838 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-07T23:18:39.951 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-blinker. 2023-10-07T23:18:39.970 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../078-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-10-07T23:18:40.009 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-07T23:18:40.147 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-07T23:18:40.312 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-oauthlib. 2023-10-07T23:18:40.327 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../079-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-10-07T23:18:40.369 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-07T23:18:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-07T23:18:40.690 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-07T23:18:40.709 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../080-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-10-07T23:18:40.748 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-07T23:18:40.877 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-07T23:18:41.170 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-websocket. 2023-10-07T23:18:41.187 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../081-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-10-07T23:18:41.227 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-10-07T23:18:41.286 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-07T23:18:41.540 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-07T23:18:41.556 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../082-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-10-07T23:18:41.598 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-10-07T23:18:41.713 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-07T23:18:41.864 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:42.153 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-07T23:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../083-ceph-mgr-k8sevents_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:42.207 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:42.341 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-07T23:18:42.556 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-07T23:18:42.574 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../084-libonig5_6.9.4-1_amd64.deb ... 2023-10-07T23:18:42.614 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-10-07T23:18:42.693 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-07T23:18:42.818 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-07T23:18:42.944 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-07T23:18:42.950 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-07T23:18:42.970 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../085-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:18:43.009 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:43.279 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-07T23:18:43.303 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package jq. 2023-10-07T23:18:43.320 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../086-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:18:43.361 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:43.605 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:43.644 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package socat. 2023-10-07T23:18:43.662 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../087-socat_1.7.3.3-2_amd64.deb ... 2023-10-07T23:18:43.702 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking socat (1.7.3.3-2) ... 2023-10-07T23:18:43.731 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-07T23:18:43.890 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-07T23:18:44.031 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package xmlstarlet. 2023-10-07T23:18:44.050 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../088-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-10-07T23:18:44.089 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-10-07T23:18:44.225 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-07T23:18:44.427 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-test. 2023-10-07T23:18:44.444 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../089-ceph-test_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:44.485 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:44.543 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-07T23:18:44.844 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-07T23:18:44.969 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-07T23:18:45.271 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-07T23:18:45.563 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-07T23:18:45.864 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-07T23:18:46.245 INFO:teuthology.orchestra.run.smithi057.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-07T23:18:46.492 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-07T23:18:46.618 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-07T23:18:46.952 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-07T23:18:47.331 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-07T23:18:47.457 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-07T23:18:47.616 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-07T23:18:47.967 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-07T23:18:48.285 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librados2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:48.469 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-07T23:18:49.405 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ceph-volume. 2023-10-07T23:18:49.423 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../090-ceph-volume_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:49.463 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:49.885 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-07T23:18:49.905 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../091-cephfs-mirror_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:49.943 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:49.992 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:50.152 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:50.278 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-07T23:18:50.288 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-colorama. 2023-10-07T23:18:50.307 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../092-python3-colorama_0.4.3-1build1_all.deb ... 2023-10-07T23:18:50.346 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2023-10-07T23:18:50.591 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-shell. 2023-10-07T23:18:50.610 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../093-cephfs-shell_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:50.649 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:50.663 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-07T23:18:50.906 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package cephfs-top. 2023-10-07T23:18:50.922 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../094-cephfs-top_17.2.6-1320-g08cac018-1focal_all.deb ... 2023-10-07T23:18:50.964 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:50.972 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-07T23:18:51.241 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package fonts-mathjax. 2023-10-07T23:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../095-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-07T23:18:51.274 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:51.433 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-07T23:18:51.785 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-07T23:18:51.801 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:51.803 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../096-libcephfs-dev_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:18:51.843 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:51.927 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-07T23:18:52.213 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package nvme-cli. 2023-10-07T23:18:52.233 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../097-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-10-07T23:18:52.262 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:52.308 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-10-07T23:18:52.388 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-07T23:18:52.651 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libjs-mathjax. 2023-10-07T23:18:52.668 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../098-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-10-07T23:18:52.709 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:18:52.723 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-07T23:18:53.795 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librbd1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:53.946 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-07T23:18:54.072 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-07T23:18:54.199 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:18:54.349 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python-natsort-doc. 2023-10-07T23:18:54.358 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-07T23:18:54.368 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../099-python-natsort-doc_7.0.1-1_all.deb ... 2023-10-07T23:18:54.406 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-10-07T23:18:54.709 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-07T23:18:54.734 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-atomicwrites. 2023-10-07T23:18:54.754 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../100-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-10-07T23:18:54.792 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-10-07T23:18:54.835 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-07T23:18:55.128 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-attr. 2023-10-07T23:18:55.147 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../101-python3-attr_19.3.0-2_all.deb ... 2023-10-07T23:18:55.178 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-10-07T23:18:55.197 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:55.323 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:55.489 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-webencodings. 2023-10-07T23:18:55.505 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-10-07T23:18:55.547 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-07T23:18:55.550 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-07T23:18:55.550 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-07T23:18:55.842 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-html5lib. 2023-10-07T23:18:55.860 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-10-07T23:18:55.900 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-10-07T23:18:55.992 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:56.118 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-07T23:18:56.270 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-10-07T23:18:56.288 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-10-07T23:18:56.328 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-07T23:18:56.428 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-07T23:18:56.788 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-07T23:18:56.806 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-packaging. 2023-10-07T23:18:56.824 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../105-python3-packaging_20.3-1_all.deb ... 2023-10-07T23:18:56.864 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-packaging (20.3-1) ... 2023-10-07T23:18:57.135 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:57.167 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pastescript. 2023-10-07T23:18:57.184 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2023-10-07T23:18:57.225 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-07T23:18:57.295 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:57.421 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:57.538 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pluggy. 2023-10-07T23:18:57.539 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:57.558 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2023-10-07T23:18:57.596 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-10-07T23:18:57.673 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:57.816 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-07T23:18:57.908 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-psutil. 2023-10-07T23:18:57.924 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-10-07T23:18:57.965 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-10-07T23:18:58.109 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-07T23:18:58.285 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-py. 2023-10-07T23:18:58.303 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-10-07T23:18:58.343 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-10-07T23:18:58.419 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:58.570 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:58.649 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-07T23:18:58.670 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-10-07T23:18:58.713 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:58.732 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-07T23:18:58.823 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-07T23:18:59.044 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-pytest. 2023-10-07T23:18:59.061 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2023-10-07T23:18:59.101 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-10-07T23:18:59.439 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package python3-simplejson. 2023-10-07T23:18:59.456 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-10-07T23:18:59.496 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-07T23:18:59.774 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-07T23:18:59.787 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:18:59.792 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-10-07T23:18:59.832 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-07T23:19:00.324 INFO:teuthology.orchestra.run.smithi057.stdout:Adding group ceph....done 2023-10-07T23:19:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package radosgw. 2023-10-07T23:19:00.650 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../114-radosgw_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:19:00.687 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:00.743 INFO:teuthology.orchestra.run.smithi057.stdout:Adding system user ceph....done 2023-10-07T23:19:00.921 INFO:teuthology.orchestra.run.smithi057.stdout:Setting system user ceph properties....done 2023-10-07T23:19:00.974 INFO:teuthology.orchestra.run.smithi057.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-07T23:19:01.127 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-07T23:19:01.436 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-07T23:19:01.875 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:02.107 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-07T23:19:02.107 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-07T23:19:02.449 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package rbd-fuse. 2023-10-07T23:19:02.467 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../115-rbd-fuse_17.2.6-1320-g08cac018-1focal_amd64.deb ... 2023-10-07T23:19:02.514 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:02.526 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:02.669 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:02.936 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-07T23:19:02.936 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-07T23:19:03.143 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-attr (19.3.0-2) ... 2023-10-07T23:19:03.440 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-10-07T23:19:03.566 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:03.566 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-10-07T23:19:03.716 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-10-07T23:19:03.919 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:03.951 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-07T23:19:04.391 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:04.574 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user cephadm....done 2023-10-07T23:19:04.629 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-07T23:19:04.629 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-07T23:19:04.735 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:19:04.862 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-10-07T23:19:05.059 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:05.205 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-07T23:19:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-07T23:19:05.326 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-07T23:19:05.498 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-10-07T23:19:05.623 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-10-07T23:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-10-07T23:19:05.769 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:06.076 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-10-07T23:19:06.249 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-07T23:19:06.249 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-07T23:19:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-10-07T23:19:06.704 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:06.771 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-10-07T23:19:06.856 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:06.998 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:07.072 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2023-10-07T23:19:07.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-07T23:19:07.229 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-07T23:19:07.373 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-10-07T23:19:07.674 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-10-07T23:19:07.675 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:07.818 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:07.944 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:07.984 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-argparse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:08.112 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:08.238 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-07T23:19:08.284 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-10-07T23:19:08.390 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-07T23:19:08.410 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2023-10-07T23:19:08.717 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-07T23:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-10-07T23:19:09.013 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-10-07T23:19:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-10-07T23:19:09.431 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-10-07T23:19:09.800 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-10-07T23:19:10.101 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-10-07T23:19:10.387 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-10-07T23:19:10.739 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-10-07T23:19:11.023 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-10-07T23:19:11.366 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-10-07T23:19:11.659 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-packaging (20.3-1) ... 2023-10-07T23:19:11.761 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-07T23:19:11.968 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-10-07T23:19:12.102 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-10-07T23:19:12.237 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-10-07T23:19:12.334 INFO:teuthology.orchestra.run.smithi057.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-07T23:19:12.336 DEBUG:teuthology.orchestra.run.smithi057:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-07T23:19:12.407 INFO:teuthology.orchestra.run.smithi057.stdout:Reading package lists... 2023-10-07T23:19:12.533 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-10-07T23:19:12.626 INFO:teuthology.orchestra.run.smithi057.stdout:Building dependency tree... 2023-10-07T23:19:12.626 INFO:teuthology.orchestra.run.smithi057.stdout:Reading state information... 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-07T23:19:12.850 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages were automatically installed and are no longer required: 2023-10-07T23:19:12.851 INFO:teuthology.orchestra.run.smithi057.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-07T23:19:12.851 INFO:teuthology.orchestra.run.smithi057.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-07T23:19:12.851 INFO:teuthology.orchestra.run.smithi057.stdout:The following additional packages will be installed: 2023-10-07T23:19:12.852 INFO:teuthology.orchestra.run.smithi057.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-07T23:19:12.852 INFO:teuthology.orchestra.run.smithi057.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-07T23:19:12.852 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-07T23:19:12.852 INFO:teuthology.orchestra.run.smithi057.stdout: usb-modeswitch-data wpasupplicant 2023-10-07T23:19:12.853 INFO:teuthology.orchestra.run.smithi057.stdout:Suggested packages: 2023-10-07T23:19:12.854 INFO:teuthology.orchestra.run.smithi057.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-07T23:19:12.854 INFO:teuthology.orchestra.run.smithi057.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-07T23:19:12.903 INFO:teuthology.orchestra.run.smithi057.stdout:The following NEW packages will be installed: 2023-10-07T23:19:12.904 INFO:teuthology.orchestra.run.smithi057.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-07T23:19:12.904 INFO:teuthology.orchestra.run.smithi057.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-07T23:19:12.904 INFO:teuthology.orchestra.run.smithi057.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-07T23:19:12.904 INFO:teuthology.orchestra.run.smithi057.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-07T23:19:12.905 INFO:teuthology.orchestra.run.smithi057.stdout:The following packages will be upgraded: 2023-10-07T23:19:12.906 INFO:teuthology.orchestra.run.smithi057.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-07T23:19:12.917 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2023-10-07T23:19:13.089 INFO:teuthology.orchestra.run.smithi057.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-07T23:19:13.090 INFO:teuthology.orchestra.run.smithi057.stdout:Need to get 10.4 MB of archives. 2023-10-07T23:19:13.090 INFO:teuthology.orchestra.run.smithi057.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-07T23:19:13.090 INFO:teuthology.orchestra.run.smithi057.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-07T23:19:13.227 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-10-07T23:19:13.595 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-10-07T23:19:13.712 INFO:teuthology.orchestra.run.smithi057.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-07T23:19:13.714 INFO:teuthology.orchestra.run.smithi057.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-07T23:19:13.715 INFO:teuthology.orchestra.run.smithi057.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-07T23:19:13.728 INFO:teuthology.orchestra.run.smithi057.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-07T23:19:13.730 INFO:teuthology.orchestra.run.smithi057.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-07T23:19:13.828 INFO:teuthology.orchestra.run.smithi057.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-07T23:19:13.831 INFO:teuthology.orchestra.run.smithi057.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-07T23:19:13.838 INFO:teuthology.orchestra.run.smithi057.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-07T23:19:13.843 INFO:teuthology.orchestra.run.smithi057.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-07T23:19:13.844 INFO:teuthology.orchestra.run.smithi057.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-07T23:19:13.989 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up socat (1.7.3.3-2) ... 2023-10-07T23:19:14.011 INFO:teuthology.orchestra.run.smithi057.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-07T23:19:14.114 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:14.280 INFO:teuthology.orchestra.run.smithi057.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-07T23:19:14.409 INFO:teuthology.orchestra.run.smithi057.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-07T23:19:14.414 INFO:teuthology.orchestra.run.smithi057.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-07T23:19:14.443 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:19:14.510 INFO:teuthology.orchestra.run.smithi057.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-07T23:19:14.515 INFO:teuthology.orchestra.run.smithi057.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-07T23:19:14.577 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-10-07T23:19:14.596 INFO:teuthology.orchestra.run.smithi057.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-07T23:19:14.671 INFO:teuthology.orchestra.run.smithi057.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-07T23:19:14.671 INFO:teuthology.orchestra.run.smithi057.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-07T23:19:14.760 INFO:teuthology.orchestra.run.smithi057.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-07T23:19:14.761 INFO:teuthology.orchestra.run.smithi057.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-07T23:19:14.955 INFO:teuthology.orchestra.run.smithi057.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-07T23:19:14.970 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-10-07T23:19:15.096 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-10-07T23:19:15.230 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-10-07T23:19:15.515 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-10-07T23:19:15.597 INFO:teuthology.orchestra.run.smithi057.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-07T23:19:15.711 INFO:teuthology.orchestra.run.smithi057.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-07T23:19:15.727 INFO:teuthology.orchestra.run.smithi057.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-07T23:19:15.729 INFO:teuthology.orchestra.run.smithi057.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-07T23:19:15.730 INFO:teuthology.orchestra.run.smithi057.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-07T23:19:15.732 INFO:teuthology.orchestra.run.smithi057.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-07T23:19:15.816 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:19:15.925 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-10-07T23:19:16.029 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-10-07T23:19:16.069 INFO:teuthology.orchestra.run.smithi057.stdout:Preconfiguring packages ... 2023-10-07T23:19:16.209 INFO:teuthology.orchestra.run.smithi057.stdout:Fetched 10.4 MB in 3s (3,700 kB/s) 2023-10-07T23:19:16.337 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-10-07T23:19:16.489 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:16.494 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:16.621 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-10-07T23:19:16.648 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:16.922 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-10-07T23:19:17.048 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-07T23:19:17.316 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-10-07T23:19:17.609 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-10-07T23:19:17.813 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-07T23:19:17.885 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-10-07T23:19:18.322 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-10-07T23:19:18.579 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-10-07T23:19:18.730 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-10-07T23:19:19.040 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-10-07T23:19:19.191 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:19.334 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:19.344 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:19.391 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-10-07T23:19:19.525 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-10-07T23:19:19.651 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-10-07T23:19:19.986 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-10-07T23:19:20.279 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librados2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:20.371 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:20.404 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-10-07T23:19:20.513 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:20.909 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-07T23:19:21.368 INFO:teuthology.orchestra.run.smithi057.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:21.373 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:21.529 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-07T23:19:21.860 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:21.994 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:19:22.027 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:22.120 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-10-07T23:19:22.152 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-07T23:19:22.480 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-10-07T23:19:22.644 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-07T23:19:22.765 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-10-07T23:19:22.807 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-07T23:19:23.049 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-10-07T23:19:23.175 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-10-07T23:19:23.344 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-07T23:19:23.362 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-07T23:19:23.402 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-07T23:19:23.534 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:23.654 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-10-07T23:19:23.739 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package dump. 2023-10-07T23:19:23.757 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-07T23:19:23.797 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking dump (0.4b46-6) ... 2023-10-07T23:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libradosstriper1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:24.110 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package indent. 2023-10-07T23:19:24.113 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-10-07T23:19:24.129 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:19:24.168 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-07T23:19:24.439 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-10-07T23:19:24.512 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-07T23:19:24.530 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-07T23:19:24.616 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-07T23:19:24.875 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-07T23:19:24.894 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-07T23:19:24.933 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-07T23:19:25.244 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-07T23:19:25.265 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:25.472 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:19:25.478 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librbd1 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:25.604 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-10-07T23:19:25.730 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-10-07T23:19:25.807 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-07T23:19:25.828 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:25.856 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-10-07T23:19:25.865 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:25.982 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rsa (4.0-3) ... 2023-10-07T23:19:26.160 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-07T23:19:26.178 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:26.218 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:26.274 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-10-07T23:19:26.400 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-10-07T23:19:26.538 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-07T23:19:26.555 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:26.587 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:26.770 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-modules-core (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:26.871 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:26.974 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-07T23:19:26.991 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-07T23:19:27.032 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-07T23:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-07T23:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-07T23:19:27.385 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-07T23:19:27.402 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-07T23:19:27.444 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-07T23:19:27.581 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libcephfs-dev (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:27.724 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-10-07T23:19:27.813 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-07T23:19:27.833 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:27.863 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:28.009 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-07T23:19:28.200 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-07T23:19:28.218 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:28.250 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:28.485 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-07T23:19:28.502 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-07T23:19:28.568 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:19:28.686 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-10-07T23:19:29.039 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rados (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:29.089 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-07T23:19:29.106 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-07T23:19:29.147 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-07T23:19:29.182 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up librgw2 (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:29.327 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-top (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:29.453 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rbd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:29.543 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package modemmanager. 2023-10-07T23:19:29.561 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:29.579 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up rbd-fuse (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:29.611 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:29.696 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-routes (2.4.1-2) ... 2023-10-07T23:19:29.990 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-10-07T23:19:30.291 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-rgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:30.292 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package wpasupplicant. 2023-10-07T23:19:30.312 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-07T23:19:30.349 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-07T23:19:30.417 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-cephfs (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:30.543 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-shell (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:30.671 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-10-07T23:19:31.382 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package network-manager. 2023-10-07T23:19:31.402 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-07T23:19:31.455 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-07T23:19:31.583 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-common (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:32.113 INFO:teuthology.orchestra.run.smithi154.stdout:Adding group ceph....done 2023-10-07T23:19:32.232 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package ppp. 2023-10-07T23:19:32.252 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-07T23:19:32.465 INFO:teuthology.orchestra.run.smithi154.stdout:Adding system user ceph....done 2023-10-07T23:19:32.512 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-07T23:19:32.641 INFO:teuthology.orchestra.run.smithi154.stdout:Setting system user ceph properties....done 2023-10-07T23:19:32.666 INFO:teuthology.orchestra.run.smithi154.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-07T23:19:32.813 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-07T23:19:32.870 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package pptp-linux. 2023-10-07T23:19:32.889 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-07T23:19:32.928 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-07T23:19:33.127 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-07T23:19:33.307 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-07T23:19:33.328 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-07T23:19:33.383 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-07T23:19:33.669 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-07T23:19:33.686 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-07T23:19:33.727 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-07T23:19:33.859 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up cephfs-mirror (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:34.104 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-07T23:19:34.104 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-07T23:19:34.139 INFO:teuthology.orchestra.run.smithi057.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-07T23:19:34.158 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-07T23:19:34.207 INFO:teuthology.orchestra.run.smithi057.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-07T23:19:34.534 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:19:34.547 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-test (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:34.673 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up radosgw (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:34.828 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-07T23:19:34.945 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-07T23:19:34.945 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-07T23:19:34.962 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-07T23:19:35.080 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-07T23:19:35.511 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-07T23:19:35.511 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-07T23:19:35.584 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-base (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:35.933 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-07T23:19:35.935 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-07T23:19:36.059 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-07T23:19:36.210 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:19:36.312 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-07T23:19:36.369 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mds (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:36.412 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-07T23:19:36.593 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-07T23:19:36.593 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-07T23:19:36.666 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-07T23:19:37.046 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:37.280 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-07T23:19:37.280 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-07T23:19:37.739 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-osd (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:37.925 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-07T23:19:38.146 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up dump (0.4b46-6) ... 2023-10-07T23:19:38.201 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-07T23:19:38.201 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-07T23:19:38.241 INFO:teuthology.orchestra.run.smithi057.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-07T23:19:38.340 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:38.465 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-07T23:19:38.592 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-07T23:19:38.658 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:38.692 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-07T23:19:38.817 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:38.869 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-07T23:19:38.986 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:19:39.110 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mon (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:39.112 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:39.272 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-07T23:19:39.439 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:19:39.590 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-07T23:19:39.609 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-07T23:19:39.609 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-07T23:19:40.079 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-cephadm (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:40.222 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout:which means they are currently configured by ifupdown: 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout:- ens1f0 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout:remove their configuration from /etc/network/interfaces. 2023-10-07T23:19:40.263 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:19:40.348 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-mgr-dashboard (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:40.422 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-07T23:19:40.491 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ceph-volume (17.2.6-1320-g08cac018-1focal) ... 2023-10-07T23:19:40.618 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-07T23:19:40.722 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-07T23:19:40.769 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-07T23:19:41.054 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-07T23:19:41.251 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-07T23:19:41.727 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:41.853 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-07T23:19:41.996 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:19:42.147 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:42.289 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:42.415 INFO:teuthology.orchestra.run.smithi057.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:42.714 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-07T23:19:42.714 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-07T23:19:43.193 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-07T23:19:43.286 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-07T23:19:43.437 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-07T23:19:43.588 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-07T23:19:43.873 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-07T23:19:43.898 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-10-07T23:19:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-07T23:19:44.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2023-10-07T23:19:44.585 INFO:teuthology.orchestra.run.smithi154.stdout:Reading package lists... 2023-10-07T23:19:44.787 INFO:teuthology.orchestra.run.smithi154.stdout:Building dependency tree... 2023-10-07T23:19:44.788 INFO:teuthology.orchestra.run.smithi154.stdout:Reading state information... 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:flex is already the newest version (2.6.4-6.2). 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:libtool-bin is already the newest version (2.4.6-14). 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2023-10-07T23:19:44.987 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages were automatically installed and are no longer required: 2023-10-07T23:19:44.988 INFO:teuthology.orchestra.run.smithi154.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-10-07T23:19:44.988 INFO:teuthology.orchestra.run.smithi154.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-07T23:19:44.988 INFO:teuthology.orchestra.run.smithi154.stdout:The following additional packages will be installed: 2023-10-07T23:19:44.988 INFO:teuthology.orchestra.run.smithi154.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2023-10-07T23:19:44.989 INFO:teuthology.orchestra.run.smithi154.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2023-10-07T23:19:44.989 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2023-10-07T23:19:44.989 INFO:teuthology.orchestra.run.smithi154.stdout: usb-modeswitch-data wpasupplicant 2023-10-07T23:19:44.990 INFO:teuthology.orchestra.run.smithi154.stdout:Suggested packages: 2023-10-07T23:19:44.990 INFO:teuthology.orchestra.run.smithi154.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2023-10-07T23:19:44.990 INFO:teuthology.orchestra.run.smithi154.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2023-10-07T23:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout:The following NEW packages will be installed: 2023-10-07T23:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2023-10-07T23:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2023-10-07T23:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2023-10-07T23:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-07T23:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout:The following packages will be upgraded: 2023-10-07T23:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2023-10-07T23:19:45.094 INFO:teuthology.orchestra.run.smithi154.stdout:6 upgraded, 23 newly installed, 0 to remove and 247 not upgraded. 2023-10-07T23:19:45.095 INFO:teuthology.orchestra.run.smithi154.stdout:Need to get 10.4 MB of archives. 2023-10-07T23:19:45.095 INFO:teuthology.orchestra.run.smithi154.stdout:After this operation, 33.5 MB of additional disk space will be used. 2023-10-07T23:19:45.095 INFO:teuthology.orchestra.run.smithi154.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.4 [1,021 kB] 2023-10-07T23:19:45.223 INFO:teuthology.orchestra.run.smithi154.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.4 [33.6 kB] 2023-10-07T23:19:45.224 INFO:teuthology.orchestra.run.smithi154.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.4 [20.0 kB] 2023-10-07T23:19:45.224 INFO:teuthology.orchestra.run.smithi154.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2023-10-07T23:19:45.229 INFO:teuthology.orchestra.run.smithi154.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2023-10-07T23:19:45.230 INFO:teuthology.orchestra.run.smithi154.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.19 [1,321 kB] 2023-10-07T23:19:45.247 INFO:teuthology.orchestra.run.smithi154.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2023-10-07T23:19:45.247 INFO:teuthology.orchestra.run.smithi154.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2023-10-07T23:19:45.249 INFO:teuthology.orchestra.run.smithi154.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2023-10-07T23:19:45.250 INFO:teuthology.orchestra.run.smithi154.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2023-10-07T23:19:45.250 INFO:teuthology.orchestra.run.smithi154.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.6 [60.6 kB] 2023-10-07T23:19:45.256 INFO:teuthology.orchestra.run.smithi154.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2023-10-07T23:19:45.256 INFO:teuthology.orchestra.run.smithi154.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2023-10-07T23:19:45.258 INFO:teuthology.orchestra.run.smithi154.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2023-10-07T23:19:45.258 INFO:teuthology.orchestra.run.smithi154.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2023-10-07T23:19:45.272 INFO:teuthology.orchestra.run.smithi154.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libndp0 amd64 1.7-0ubuntu1 [10.9 kB] 2023-10-07T23:19:45.273 INFO:teuthology.orchestra.run.smithi154.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.3 [370 kB] 2023-10-07T23:19:45.278 INFO:teuthology.orchestra.run.smithi154.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2023-10-07T23:19:45.288 INFO:teuthology.orchestra.run.smithi154.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2023-10-07T23:19:45.288 INFO:teuthology.orchestra.run.smithi154.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.19 [1,584 kB] 2023-10-07T23:19:45.302 INFO:teuthology.orchestra.run.smithi154.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2023-10-07T23:19:45.302 INFO:teuthology.orchestra.run.smithi154.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2023-10-07T23:19:45.309 INFO:teuthology.orchestra.run.smithi154.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.3 [1,183 kB] 2023-10-07T23:19:45.319 INFO:teuthology.orchestra.run.smithi154.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.3 [1,855 kB] 2023-10-07T23:19:45.335 INFO:teuthology.orchestra.run.smithi154.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2023-10-07T23:19:45.337 INFO:teuthology.orchestra.run.smithi154.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2023-10-07T23:19:45.338 INFO:teuthology.orchestra.run.smithi154.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2023-10-07T23:19:45.338 INFO:teuthology.orchestra.run.smithi154.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2023-10-07T23:19:45.338 INFO:teuthology.orchestra.run.smithi154.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2023-10-07T23:19:45.655 INFO:teuthology.orchestra.run.smithi154.stdout:Preconfiguring packages ... 2023-10-07T23:19:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:Fetched 10.4 MB in 0s (35.8 MB/s) 2023-10-07T23:19:46.083 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:46.088 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:46.251 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking util-linux (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:47.415 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up util-linux (2.34-0.1ubuntu9.4) ... 2023-10-07T23:19:47.742 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-07T23:19:47.842 INFO:teuthology.orchestra.run.smithi057.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-07T23:19:48.723 INFO:teuthology.orchestra.run.smithi057.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-07T23:19:48.726 DEBUG:teuthology.parallel:result is None 2023-10-07T23:19:48.862 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:48.865 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:48.990 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:49.396 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.4_amd64.deb ... 2023-10-07T23:19:49.547 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.4) over (2.34-0.1ubuntu9.3) ... 2023-10-07T23:19:49.912 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-07T23:19:50.376 INFO:teuthology.orchestra.run.smithi154.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152664 files and directories currently installed.) 2023-10-07T23:19:50.381 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:50.513 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-07T23:19:50.910 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:51.043 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2023-10-07T23:19:51.557 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-07T23:19:51.690 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.19) over (1.1.1f-1ubuntu2.16) ... 2023-10-07T23:19:52.160 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-07T23:19:52.179 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2023-10-07T23:19:52.210 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2023-10-07T23:19:52.597 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package dump. 2023-10-07T23:19:52.615 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2023-10-07T23:19:52.655 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking dump (0.4b46-6) ... 2023-10-07T23:19:52.942 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package indent. 2023-10-07T23:19:52.959 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2023-10-07T23:19:53.000 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-07T23:19:53.278 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-07T23:19:53.295 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2023-10-07T23:19:53.328 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2023-10-07T23:19:53.648 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-07T23:19:53.669 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.6_amd64.deb ... 2023-10-07T23:19:53.706 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-07T23:19:54.026 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-07T23:19:54.043 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2023-10-07T23:19:54.084 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:19:54.412 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-07T23:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:54.723 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-07T23:19:54.740 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:54.856 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:19:55.645 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-07T23:19:55.665 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:55.703 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:56.056 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-07T23:19:56.076 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1_amd64.deb ... 2023-10-07T23:19:56.114 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-07T23:19:56.492 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-07T23:19:56.513 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-07T23:19:56.550 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-07T23:19:56.929 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-07T23:19:56.945 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:56.987 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:57.306 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-07T23:19:57.323 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:57.364 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:19:57.634 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-07T23:19:57.651 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.19_amd64.deb ... 2023-10-07T23:19:57.692 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:19:58.162 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-07T23:19:58.182 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2023-10-07T23:19:58.220 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2023-10-07T23:19:58.607 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package modemmanager. 2023-10-07T23:19:58.624 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2023-10-07T23:19:58.676 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:19:59.305 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package wpasupplicant. 2023-10-07T23:19:59.322 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.3_amd64.deb ... 2023-10-07T23:19:59.363 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-07T23:19:59.902 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager. 2023-10-07T23:19:59.919 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.3_amd64.deb ... 2023-10-07T23:19:59.984 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager (1.22.10-1ubuntu2.3) ... 2023-10-07T23:20:00.832 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package ppp. 2023-10-07T23:20:00.853 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2023-10-07T23:20:01.162 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-07T23:20:01.524 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package pptp-linux. 2023-10-07T23:20:01.540 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2023-10-07T23:20:01.581 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2023-10-07T23:20:01.877 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-07T23:20:01.893 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2023-10-07T23:20:01.944 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2023-10-07T23:20:02.196 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-07T23:20:02.214 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2023-10-07T23:20:02.255 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2023-10-07T23:20:02.641 INFO:teuthology.orchestra.run.smithi154.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-07T23:20:02.662 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2023-10-07T23:20:02.710 INFO:teuthology.orchestra.run.smithi154.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-07T23:20:03.011 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:20:03.339 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2023-10-07T23:20:03.465 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2023-10-07T23:20:03.590 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.3) ... 2023-10-07T23:20:04.020 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-07T23:20:04.020 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-07T23:20:04.502 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.4) ... 2023-10-07T23:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2023-10-07T23:20:04.754 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.19) ... 2023-10-07T23:20:04.880 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2023-10-07T23:20:05.006 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.3) ... 2023-10-07T23:20:05.158 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2023-10-07T23:20:06.147 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2023-10-07T23:20:06.652 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up dump (0.4b46-6) ... 2023-10-07T23:20:06.743 INFO:teuthology.orchestra.run.smithi154.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-07T23:20:06.808 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:20:06.934 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.6) ... 2023-10-07T23:20:07.060 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2023-10-07T23:20:07.186 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2023-10-07T23:20:07.380 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1) ... 2023-10-07T23:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:20:07.665 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:20:07.791 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2023-10-07T23:20:07.959 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:20:08.085 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager (1.22.10-1ubuntu2.3) ... 2023-10-07T23:20:08.822 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:08.822 INFO:teuthology.orchestra.run.smithi154.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-07T23:20:08.823 INFO:teuthology.orchestra.run.smithi154.stdout:which means they are currently configured by ifupdown: 2023-10-07T23:20:08.823 INFO:teuthology.orchestra.run.smithi154.stdout:- enp3s0f1 2023-10-07T23:20:08.823 INFO:teuthology.orchestra.run.smithi154.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-07T23:20:08.823 INFO:teuthology.orchestra.run.smithi154.stdout:remove their configuration from /etc/network/interfaces. 2023-10-07T23:20:08.823 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:08.988 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-07T23:20:09.323 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-07T23:20:09.848 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-07T23:20:10.330 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2023-10-07T23:20:10.474 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2023-10-07T23:20:10.616 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2023-10-07T23:20:10.734 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:20:10.835 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2023-10-07T23:20:10.936 INFO:teuthology.orchestra.run.smithi154.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2023-10-07T23:20:11.213 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-07T23:20:11.213 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-07T23:20:11.722 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2023-10-07T23:20:11.832 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-07T23:20:11.949 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-10-07T23:20:12.101 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-10-07T23:20:12.394 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-10-07T23:20:16.205 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2023-10-07T23:20:16.322 INFO:teuthology.orchestra.run.smithi154.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2023-10-07T23:20:17.534 INFO:teuthology.orchestra.run.smithi154.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-07T23:20:17.534 DEBUG:teuthology.parallel:result is None 2023-10-07T23:20:17.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:20:17.679 DEBUG:teuthology.orchestra.run.smithi057:> dpkg-query -W -f '${Version}' ceph 2023-10-07T23:20:17.703 INFO:teuthology.orchestra.run.smithi057.stdout:17.2.6-1320-g08cac018-1focal 2023-10-07T23:20:17.704 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320-g08cac018-1focal 2023-10-07T23:20:17.704 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320-g08cac018-1focal is installed. 2023-10-07T23:20:17.704 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-07T23:20:17.843 DEBUG:teuthology.orchestra.run.smithi154:> dpkg-query -W -f '${Version}' ceph 2023-10-07T23:20:17.868 INFO:teuthology.orchestra.run.smithi154.stdout:17.2.6-1320-g08cac018-1focal 2023-10-07T23:20:17.868 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320-g08cac018-1focal 2023-10-07T23:20:17.868 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320-g08cac018-1focal is installed. 2023-10-07T23:20:17.869 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-07T23:20:17.869 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:17.869 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T23:20:17.887 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:17.887 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-07T23:20:17.900 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-07T23:20:17.901 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:17.901 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T23:20:17.945 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T23:20:17.955 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:17.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2023-10-07T23:20:17.970 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-07T23:20:17.980 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-07T23:20:17.980 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:17.980 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T23:20:17.995 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T23:20:18.053 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:18.054 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-07T23:20:18.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-07T23:20:18.126 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-07T23:20:18.126 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:18.126 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T23:20:18.142 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T23:20:18.154 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:18.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2023-10-07T23:20:18.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-07T23:20:18.226 INFO:teuthology.run_tasks:Running task ceph... 2023-10-07T23:20:18.287 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-07T23:20:18.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /var/log/ceph 2023-10-07T23:20:18.290 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2023-10-07T23:20:18.304 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-07T23:20:18.304 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-07T23:20:18.306 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-07T23:20:18.363 INFO:tasks.ceph:Creating extra log directories... 2023-10-07T23:20:18.363 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-07T23:20:18.365 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-07T23:20:18.389 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-07T23:20:18.389 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-07T23:20:18.389 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-07T23:20:18.390 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:18.390 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-07T23:20:18.428 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-07T23:20:18.467 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:18.467 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-07T23:20:18.474 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-07T23:20:18.562 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-07T23:20:18.562 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-07T23:20:18.562 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7418060', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006', 'targets': {'smithi057.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-07_22:15:20', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-07T23:20:18.563 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-07T23:20:18.565 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-07T23:20:18.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m0777 -- /var/run/ceph 2023-10-07T23:20:18.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2023-10-07T23:20:18.627 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:18.627 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T23:20:18.633 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T23:20:18.634 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 360 Links: 1 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-07 23:15:28.963717681 +0000 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-07 23:15:28.671724416 +0000 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-07 23:15:28.671724416 +0000 2023-10-07T23:20:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-07T23:20:18.642 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T23:20:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-07T23:20:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-07T23:20:18.652 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000530131 s, 966 kB/s 2023-10-07T23:20:18.653 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T23:20:18.702 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 368 Links: 1 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-07 23:15:29.443706610 +0000 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-07 23:15:29.159713160 +0000 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-07 23:15:29.159713160 +0000 2023-10-07T23:20:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-07T23:20:18.710 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T23:20:18.724 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-07T23:20:18.725 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-07T23:20:18.725 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000483193 s, 1.1 MB/s 2023-10-07T23:20:18.725 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T23:20:18.774 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2023-10-07T23:20:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T23:20:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:18.781 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-10-07T23:20:18.781 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:18.782 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-07 23:15:29.935695260 +0000 2023-10-07T23:20:18.782 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-07 23:15:29.647701903 +0000 2023-10-07T23:20:18.782 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-07 23:15:29.647701903 +0000 2023-10-07T23:20:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-07T23:20:18.782 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T23:20:18.793 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-07T23:20:18.793 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-07T23:20:18.794 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.00046069 s, 1.1 MB/s 2023-10-07T23:20:18.795 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T23:20:18.846 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2023-10-07T23:20:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 379 Links: 1 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2023-10-07 23:15:30.599679944 +0000 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2023-10-07 23:15:30.327686218 +0000 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2023-10-07 23:15:30.327686218 +0000 2023-10-07T23:20:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: - 2023-10-07T23:20:18.854 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T23:20:18.865 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2023-10-07T23:20:18.865 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2023-10-07T23:20:18.865 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000557015 s, 919 kB/s 2023-10-07T23:20:18.866 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T23:20:18.918 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:18.919 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:18.919 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2023-10-07T23:20:18.925 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-07T23:20:18.926 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 359 Links: 1 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-07 23:15:28.102086809 +0000 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-07 23:15:27.822093264 +0000 2023-10-07T23:20:18.933 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-07 23:15:27.822093264 +0000 2023-10-07T23:20:18.934 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-07T23:20:18.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-07T23:20:18.989 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-07T23:20:18.989 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-07T23:20:18.989 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000403708 s, 1.3 MB/s 2023-10-07T23:20:18.991 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-07T23:20:18.999 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2023-10-07T23:20:19.005 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-07T23:20:19.005 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 365 Links: 1 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-07 23:15:28.614075007 +0000 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-07 23:15:28.330081554 +0000 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-07 23:15:28.330081554 +0000 2023-10-07T23:20:19.006 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-07T23:20:19.006 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-07T23:20:19.061 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-07T23:20:19.061 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-07T23:20:19.061 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000394461 s, 1.3 MB/s 2023-10-07T23:20:19.062 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-07T23:20:19.071 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2023-10-07T23:20:19.076 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 374 Links: 1 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-07 23:15:29.110063575 +0000 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-07 23:15:28.830070030 +0000 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-07 23:15:28.830070030 +0000 2023-10-07T23:20:19.077 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-07T23:20:19.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-07T23:20:19.133 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-07T23:20:19.133 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-07T23:20:19.133 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000407724 s, 1.3 MB/s 2023-10-07T23:20:19.134 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-07T23:20:19.142 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-10-07 23:15:29.614051958 +0000 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-10-07 23:15:29.326058596 +0000 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-10-07 23:15:29.326058596 +0000 2023-10-07T23:20:19.148 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2023-10-07T23:20:19.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-07T23:20:19.205 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-10-07T23:20:19.205 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-10-07T23:20:19.206 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000422454 s, 1.2 MB/s 2023-10-07T23:20:19.207 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-07T23:20:19.215 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:19.215 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-07T23:20:19.215 INFO:tasks.ceph:Generating config... 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] debug client = 20 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] debug ms = 1 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] pid file = 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-07T23:20:19.217 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-07T23:20:19.217 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-07T23:20:19.217 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-07T23:20:19.217 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-07T23:20:19.217 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-07T23:20:19.218 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-07T23:20:19.218 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-07T23:20:19.219 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-07T23:20:19.220 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-07T23:20:19.220 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-07T23:20:19.220 INFO:tasks.ceph:Setting up mon.a... 2023-10-07T23:20:19.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-07T23:20:19.255 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.keyring 2023-10-07T23:20:19.258 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-07T23:20:19.293 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T23:20:19.347 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.57'), ('mon.b', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]')] 2023-10-07T23:20:19.348 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.57,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-07T23:20:19.348 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:19.349 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-07T23:20:19.353 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.57 --add b 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:19.390 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:19.390 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: generated fsid aaf09e93-912b-470e-9df7-bb9a606f50e4 2023-10-07T23:20:19.390 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = octopus 2023-10-07T23:20:19.390 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 0 2023-10-07T23:20:19.390 INFO:teuthology.orchestra.run.smithi057.stdout:fsid aaf09e93-912b-470e-9df7-bb9a606f50e4 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:last_changed 2023-10-07T23:20:19.389438+0000 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:created 2023-10-07T23:20:19.389438+0000 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:min_mon_release 15 (octopus) 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:election_strategy: 1 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:1: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2023-10-07T23:20:19.391 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-07T23:20:19.392 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-07T23:20:19.396 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aaf09e93-912b-470e-9df7-bb9a606f50e4... 2023-10-07T23:20:19.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-07T23:20:19.399 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-07T23:20:19.429 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: chdir = "" 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: auth supported = cephx 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: filestore xattr use omap = true 2023-10-07T23:20:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: mon clock drift allowed = 1.000 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: osd crush chooseleaf type = 0 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: auth debug = true 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on old message = true 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: ms die on bug = true 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: mon pg warn max object skew = 0 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: # disable pg_autoscaler by default for new pools 2023-10-07T23:20:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default size = 2 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow primary affinity = true 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd allow pg remap = true 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on legacy crush tunables = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on crush straw calc version zero = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on no sortbitwise = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on osd down out interval zero = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon warn on too few osds = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-07T23:20:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: mon_allow_pool_size_one = true 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: osd default data pool replay window = 5 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: mon allow pool delete = true 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: mon cluster log file level = debug 2023-10-07T23:20:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: debug asserts on shutdown = true 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: mon health detail to clog = false 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: mon host = "172.21.15.57,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = aaf09e93-912b-470e-9df7-bb9a606f50e4 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout:[osd] 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: osd journal size = 100 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub load threshold = 5.0 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: osd scrub max interval = 600 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: osd mclock profile = high_recovery_ops 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: osd recover clone overlap = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd recovery max chunk = 1048576 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug shutdown = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug op order = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug verify stray on activate = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd open classes on start = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug pg log writeout = true 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd deep scrub update digest min age = 30 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: osd map max advance = 10 2023-10-07T23:20:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: journal zero on create = true 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: filestore ondisk finisher threads = 3 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: filestore apply finisher threads = 3 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bdev debug aio = true 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: osd debug misdirected ops = true 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bdev async discard = True 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bdev enable discard = True 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore allocator = bitmap 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore block size = 96636764160 2023-10-07T23:20:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: bluestore fsck on mount = True 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluefs = 1/20 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: debug bluestore = 1/20 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: debug osd = 20 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: debug rocksdb = 4/10 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd backfillfull_ratio = 0.85 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd full ratio = 0.9 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd nearfull ratio = 0.8 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: osd failsafe full ratio = 0.95 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: osd objectstore = bluestore 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: osd op complaint time = 180 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.437 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr] 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug mgr = 20 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: mgr/telemetry/nag = false 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout:[mon] 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug mon = 20 2023-10-07T23:20:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: debug paxos = 20 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: debug auth = 20 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon data avail warn = 5 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon mgr mkfs grace = 240 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min pgs per osd = 4 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd reporter subtree level = osd 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon osd prime pg temp = true 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: mon reweight min bytes per osd = 10 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: auth mon ticket ttl = 660 # 11m 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: auth service ticket ttl = 240 # 4m 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: # don't complain about insecure global_id in the test suite 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: # 1m isn't quite enough 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: mon_down_mkfs_grace = 2m 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: mon_warn_on_filestore_osds = false 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: mon op complaint time = 120 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout:[client] 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: rgw cache enabled = true 2023-10-07T23:20:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable ops log = true 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: rgw enable usage log = true 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: client mount timeout = 600 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: debug client = 20 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: pid file = "" 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: rados mon op timeout = 900 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: rados osd op timeout = 900 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.a] 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.b] 2023-10-07T23:20:19.441 INFO:teuthology.orchestra.run.smithi057.stdout:[mon.c] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout:[mds] 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: debug mds = 20 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: debug ms = 1 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: mds debug frag = True 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: mds debug scatterstat = True 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: mds op complaint time = 180 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: mds verify scatter = True 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: osd op complaint time = 180 2023-10-07T23:20:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: rados mon op timeout = 900 2023-10-07T23:20:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: rados osd op timeout = 900 2023-10-07T23:20:19.443 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-10-07T23:20:19.443 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-10-07T23:20:19.443 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-10-07T23:20:19.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-10-07T23:20:19.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.57,172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790]" 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = aaf09e93-912b-470e-9df7-bb9a606f50e4 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-10-07T23:20:19.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2023-10-07T23:20:19.449 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-10-07T23:20:19.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.451 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.452 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-10-07T23:20:19.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-07T23:20:19.454 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-10-07T23:20:19.455 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2023-10-07T23:20:19.456 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-10-07T23:20:19.456 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2023-10-07T23:20:19.456 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-10-07T23:20:19.456 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-10-07T23:20:19.456 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-10-07T23:20:19.457 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-07T23:20:19.457 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-07T23:20:19.547 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-07T23:20:19.547 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:19.547 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-07T23:20:19.552 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:19.552 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-07T23:20:19.558 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:20:19.559 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:19.559 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-07T23:20:19.559 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T23:20:19.582 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:19.583 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:19.632 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:20:19.632 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:19.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-07T23:20:19.633 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-07T23:20:19.660 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:19.660 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:19.666 INFO:tasks.ceph:Setting up mon nodes... 2023-10-07T23:20:19.667 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-07T23:20:19.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-07T23:20:19.714 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-07T23:20:19.716 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-07T23:20:19.760 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-07T23:20:19.763 INFO:tasks.ceph:Setting up mds nodes... 2023-10-07T23:20:19.763 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-07T23:20:19.811 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-07T23:20:19.813 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-07T23:20:19.826 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-07T23:20:19.863 INFO:teuthology.orchestra.run.smithi057.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-07T23:20:19.865 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-07T23:20:19.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-07T23:20:19.924 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-07T23:20:19.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-07T23:20:19.938 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-07T23:20:19.977 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-07T23:20:19.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-07T23:20:20.037 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-07T23:20:20.037 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-07T23:20:20.075 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-07T23:20:20.088 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-07T23:20:20.120 INFO:teuthology.orchestra.run.smithi057.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-07T23:20:20.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-07T23:20:20.169 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-07T23:20:20.182 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-07T23:20:20.263 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-07T23:20:20.275 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-07T23:20:20.275 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-07T23:20:20.275 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-07T23:20:20.292 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:20.292 INFO:tasks.ceph:role: osd.0 2023-10-07T23:20:20.292 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:20:20.292 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-07T23:20:21.676 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:21.676 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:21.676 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:21.676 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:21.677 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:21.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-10-07T23:20:21.678 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-07T23:20:21.702 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-07T23:20:21.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:21.713 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:21.713 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-07T23:20:21.768 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:21.769 INFO:tasks.ceph:role: osd.1 2023-10-07T23:20:21.769 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:20:21.769 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:23.132 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:23.133 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:23.134 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-10-07T23:20:23.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-07T23:20:23.166 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-07T23:20:23.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:23.217 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:23.217 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-07T23:20:23.228 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:23.228 INFO:tasks.ceph:role: osd.2 2023-10-07T23:20:23.228 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:20:23.229 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:24.586 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:24.587 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:24.589 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-10-07T23:20:24.589 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-07T23:20:24.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-07T23:20:24.666 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:24.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-07T23:20:24.683 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:24.683 INFO:tasks.ceph:role: osd.3 2023-10-07T23:20:24.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:20:24.683 DEBUG:teuthology.orchestra.run.smithi057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout: = reflink=1 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:26.025 INFO:teuthology.orchestra.run.smithi057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:26.026 INFO:teuthology.orchestra.run.smithi057.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:26.026 INFO:teuthology.orchestra.run.smithi057.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:26.026 INFO:teuthology.orchestra.run.smithi057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:26.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi057.front.sepia.ceph.com -o noatime 2023-10-07T23:20:26.027 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-07T23:20:26.060 DEBUG:teuthology.orchestra.run.smithi057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-07T23:20:26.120 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:26.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:26.121 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:26.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.234+0000 7fe5a1b2e3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-07T23:20:26.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.234+0000 7fe5a1b2e3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-07T23:20:26.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.234+0000 7fe5a1b2e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T23:20:26.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.234+0000 7fe5a1b2e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T23:20:26.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.234+0000 7fe5a1b2e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-07T23:20:26.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.238+0000 7fe5a1b2e3c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-07T23:20:26.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:26.810+0000 7fe5a1b2e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:27.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:27.590+0000 7fe5a1b2e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:28.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:28.382+0000 7fe5a1b2e3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:29.259 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-07T23:20:29.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:29.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-07T23:20:29.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-07T23:20:29.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T23:20:29.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T23:20:29.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-07T23:20:29.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.354+0000 7fd7653803c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-07T23:20:29.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:29.918+0000 7fd7653803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:30.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:30.706+0000 7fd7653803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:31.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:31.494+0000 7fd7653803c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:32.363 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-07T23:20:32.378 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:32.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-07T23:20:32.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-07T23:20:32.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T23:20:32.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T23:20:32.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-07T23:20:32.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.410+0000 7f0f4c58b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-07T23:20:32.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:32.970+0000 7f0f4c58b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:33.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:33.766+0000 7f0f4c58b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:34.549 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:34.542+0000 7f0f4c58b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:35.418 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-07T23:20:35.432 DEBUG:teuthology.orchestra.run.smithi057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:35.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.510+0000 7f1481fa73c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-07T23:20:35.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.510+0000 7f1481fa73c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-07T23:20:35.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.510+0000 7f1481fa73c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T23:20:35.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.510+0000 7f1481fa73c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T23:20:35.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.510+0000 7f1481fa73c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-07T23:20:35.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:35.514+0000 7f1481fa73c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-07T23:20:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:36.082+0000 7f1481fa73c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:36.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:36.866+0000 7f1481fa73c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:37.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:37.650+0000 7f1481fa73c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:38.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-07T23:20:38.566 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-07T23:20:38.583 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:38.583 INFO:tasks.ceph:role: osd.4 2023-10-07T23:20:38.583 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:20:38.583 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-07T23:20:39.153 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:39.153 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:39.153 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:39.153 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:39.154 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:39.156 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-07T23:20:39.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-07T23:20:39.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-07T23:20:39.251 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:39.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:39.252 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-07T23:20:39.312 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:39.312 INFO:tasks.ceph:role: osd.5 2023-10-07T23:20:39.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:20:39.312 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-07T23:20:39.825 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:39.825 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:39.825 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:39.825 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2023-10-07T23:20:39.825 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:39.826 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:39.826 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:39.826 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:39.826 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:39.826 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:39.827 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-07T23:20:39.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-07T23:20:39.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-07T23:20:39.869 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:39.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:39.870 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-07T23:20:39.927 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:39.927 INFO:tasks.ceph:role: osd.6 2023-10-07T23:20:39.927 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:20:39.927 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:40.391 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:40.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-07T23:20:40.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-07T23:20:40.455 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-07T23:20:40.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:40.468 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:40.468 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-07T23:20:40.528 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-07T23:20:40.528 INFO:tasks.ceph:role: osd.7 2023-10-07T23:20:40.528 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:20:40.528 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-07T23:20:41.037 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-07T23:20:41.039 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-10-07T23:20:41.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-07T23:20:41.103 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-07T23:20:41.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:20:41.116 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: /sbin/restorecon: command not found 2023-10-07T23:20:41.116 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:41.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.241+0000 7f4cfc3a23c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-07T23:20:41.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.245+0000 7f4cfc3a23c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-07T23:20:41.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.245+0000 7f4cfc3a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T23:20:41.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.245+0000 7f4cfc3a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T23:20:41.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.245+0000 7f4cfc3a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-07T23:20:41.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:41.253+0000 7f4cfc3a23c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-07T23:20:42.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:42.057+0000 7f4cfc3a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:42.841+0000 7f4cfc3a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:43.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:43.617+0000 7f4cfc3a23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:44.473 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-07T23:20:44.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:44.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.521+0000 7f7a8fbb53c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-07T23:20:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.521+0000 7f7a8fbb53c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-07T23:20:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.521+0000 7f7a8fbb53c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T23:20:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.521+0000 7f7a8fbb53c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T23:20:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.521+0000 7f7a8fbb53c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-07T23:20:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:44.525+0000 7f7a8fbb53c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-07T23:20:45.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:45.085+0000 7f7a8fbb53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:45.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:45.865+0000 7f7a8fbb53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:46.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:46.653+0000 7f7a8fbb53c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:47.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-07T23:20:47.544 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:47.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-07T23:20:47.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-07T23:20:47.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T23:20:47.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T23:20:47.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-07T23:20:47.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:47.625+0000 7fb4cbe1b3c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-07T23:20:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:48.193+0000 7fb4cbe1b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:48.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:20:48.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:20:48.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:48.981+0000 7fb4cbe1b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:49.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:49.757+0000 7fb4cbe1b3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:50.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-07T23:20:50.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:50.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-07T23:20:50.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-07T23:20:50.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T23:20:50.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T23:20:50.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-07T23:20:50.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:50.725+0000 7f8745cf23c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-07T23:20:51.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:51.289+0000 7f8745cf23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:52.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:52.077+0000 7f8745cf23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:52.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:20:52.865+0000 7f8745cf23c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-07T23:20:53.731 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-07T23:20:53.743 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-07T23:20:53.743 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.744 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-07T23:20:53.760 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.760 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-07T23:20:53.772 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-07T23:20:53.783 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.783 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-07T23:20:53.797 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.797 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-07T23:20:53.808 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.808 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-07T23:20:53.865 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:53.865 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-07T23:20:53.882 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-07T23:20:53.898 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.898 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-07T23:20:53.954 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-07T23:20:53.966 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-07T23:20:53.977 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-07T23:20:53.991 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:53.991 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-07T23:20:54.050 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:54.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-07T23:20:54.061 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:54.061 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-07T23:20:54.068 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:20:54.068 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-07T23:20:54.115 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:54.116 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-07T23:20:54.122 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:20:54.122 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-07T23:20:54.169 INFO:tasks.ceph:Adding keys to all mons... 2023-10-07T23:20:54.169 DEBUG:teuthology.orchestra.run.smithi057:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-07T23:20:54.171 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFlDPSDKhAALtUKYEj0TqzdoaXF8L+f/Q== 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFl4EJHMBAAHkCAWP6+9FdWp9FusnpX3w== 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFlooFlMxAAG+NVHxNzdNg5cRjAtnAOLQ== 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2023-10-07T23:20:54.183 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC65yFlqv5KDhAA25InypbQtn4sjojMQ1q4UA== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC95yFl3NxlFRAAZBqGACWZ4uyN0fXNtNtuYQ== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDA5yFlYBmuGBAAgcTTcujrs1CIHWZWM1o2VQ== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDD5yFlwgfQHhAAZMmLKVbUCsBebAOEy1pBcA== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFlB/VhLRAAld/Nnm46+mosax2vvzWOJQ== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFldG8xNxAAj8EFNonJE0X5pMyB0izMxQ== 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-10-07T23:20:54.184 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQCz5yFloYdROhAA1y7ZdlNsVUsuaarTEcolVQ== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDJ5yFld6uuDhAARh1yNxrm4Ub+VTrik7tLXg== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDM5yFlaPZvHxAAwYp3I5ZpOy5S1nBrajKpew== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDP5yFlGvyDJRAAOFvXIQkB+6Q0YyaQ9McSYA== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQDS5yFl445yKxAAh/wYkbCQUJy+nmP4S1lsZA== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC05yFlzElqBBAAvQsOKVpmlkOCrZOBdS/Mng== 2023-10-07T23:20:54.185 INFO:teuthology.orchestra.run.smithi154.stdout:[client.2] 2023-10-07T23:20:54.186 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC05yFlZJ0YBxAAbk4yS+Sl1PXtOnTxzFikDg== 2023-10-07T23:20:54.186 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2023-10-07T23:20:54.186 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC05yFlydslChAAcyKLUVIvuQ8YjyJ+J0Lepw== 2023-10-07T23:20:54.186 INFO:teuthology.orchestra.run.smithi154.stdout:[client.3] 2023-10-07T23:20:54.186 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQC05yFlS/3JDxAADLDbTFyercFRlld7VfwQEA== 2023-10-07T23:20:54.187 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.y] 2023-10-07T23:20:54.187 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFlDPSDKhAALtUKYEj0TqzdoaXF8L+f/Q== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFl4EJHMBAAHkCAWP6+9FdWp9FusnpX3w== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFlooFlMxAAG+NVHxNzdNg5cRjAtnAOLQ== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.0] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC65yFlqv5KDhAA25InypbQtn4sjojMQ1q4UA== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.1] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC95yFl3NxlFRAAZBqGACWZ4uyN0fXNtNtuYQ== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.2] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDA5yFlYBmuGBAAgcTTcujrs1CIHWZWM1o2VQ== 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.3] 2023-10-07T23:20:54.188 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDD5yFlwgfQHhAAZMmLKVbUCsBebAOEy1pBcA== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[mgr.x] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFlB/VhLRAAld/Nnm46+mosax2vvzWOJQ== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFldG8xNxAAj8EFNonJE0X5pMyB0izMxQ== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQCz5yFloYdROhAA1y7ZdlNsVUsuaarTEcolVQ== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.4] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDJ5yFld6uuDhAARh1yNxrm4Ub+VTrik7tLXg== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.5] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDM5yFlaPZvHxAAwYp3I5ZpOy5S1nBrajKpew== 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.6] 2023-10-07T23:20:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDP5yFlGvyDJRAAOFvXIQkB+6Q0YyaQ9McSYA== 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout:[osd.7] 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQDS5yFl445yKxAAh/wYkbCQUJy+nmP4S1lsZA== 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC05yFlzElqBBAAvQsOKVpmlkOCrZOBdS/Mng== 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout:[client.2] 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC05yFlZJ0YBxAAbk4yS+Sl1PXtOnTxzFikDg== 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout:[client.1] 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC05yFlydslChAAcyKLUVIvuQ8YjyJ+J0Lepw== 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout:[client.3] 2023-10-07T23:20:54.190 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQC05yFlS/3JDxAADLDbTFyercFRlld7VfwQEA== 2023-10-07T23:20:54.191 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T23:20:54.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T23:20:54.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.366 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.369 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.416 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.461 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.548 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.551 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.594 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.596 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T23:20:54.683 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-07T23:20:54.730 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.732 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.779 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-07T23:20:54.826 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.910 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:54.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:55.043 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:55.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-07T23:20:55.090 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.179 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.228 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.272 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.275 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-07T23:20:55.319 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-07T23:20:55.319 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-07T23:20:55.377 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T23:20:55.452 DEBUG:teuthology.orchestra.run.smithi057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-07T23:20:55.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-07T23:20:55.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T23:20:55.539 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-07T23:20:55.552 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-07T23:20:55.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-07T23:20:55.616 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-07T23:20:55.627 DEBUG:teuthology.orchestra.run.smithi057:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:55.630 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-07T23:20:55.634 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-07T23:20:55.635 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-07T23:20:55.635 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-07T23:20:55.679 INFO:tasks.ceph.mon.a:Started 2023-10-07T23:20:55.679 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-07T23:20:55.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-07T23:20:55.681 INFO:tasks.ceph.mon.b:Started 2023-10-07T23:20:55.682 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-07T23:20:55.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-07T23:20:55.684 INFO:tasks.ceph.mon.c:Started 2023-10-07T23:20:55.684 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-07T23:20:55.685 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-07T23:20:55.685 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-07T23:20:55.687 INFO:tasks.ceph.mgr.y:Started 2023-10-07T23:20:55.687 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-07T23:20:55.687 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-07T23:20:55.689 INFO:tasks.ceph.mgr.x:Started 2023-10-07T23:20:55.689 INFO:tasks.ceph:Setting crush tunables to default 2023-10-07T23:20:55.690 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph osd crush tunables default 2023-10-07T23:20:55.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 Processor -- start 2023-10-07T23:20:55.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 -- start start 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c155fa0 con 0x7f8e4c155790 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c06a070 con 0x7f8e4c155b60 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8e4c155790 0x7f8e4c0c8600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:42646/0 (socket says 172.21.15.57:42646) 2023-10-07T23:20:55.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4b7fe700 1 -- 172.21.15.57:0/2167987857 learned_addr learned my addr 172.21.15.57:0/2167987857 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c159ca0 con 0x7f8e4c069c20 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:20:55.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-07T23:20:55.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:55.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:55.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:55.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-07T23:20:56.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:56.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-07T23:20:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:57.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:57.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-07T23:20:58.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 msgr2=0x7f8e4c15a130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.154:3300/0 conn(0x7f8e4c069c20 0x7f8e4c15a130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 -- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 msgr2=0x7f8e4c0cbf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 --2- 172.21.15.57:0/2167987857 >> v2:172.21.15.57:3300/0 conn(0x7f8e4c155b60 0x7f8e4c0cbf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 -- 172.21.15.57:0/2167987857 >> v1:172.21.15.57:6789/0 conn(0x7f8e4c155790 legacy=0x7f8e4c0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 -- 172.21.15.57:0/2167987857 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c159670 con 0x7f8e2c0036e0 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e497fa700 1 -- 172.21.15.57:0/2167987857 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c06a070 con 0x7f8e2c007300 2023-10-07T23:20:58.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:58.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:58.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:58.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:58.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.790+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-07T23:20:58.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:58.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.994+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:58.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.994+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:58.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:58.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:58.994+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-07T23:20:59.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:59.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:20:59.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:59.398+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:20:59.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:59.398+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:20:59.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:59.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:20:59.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:20:59.398+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-07T23:21:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.202+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-07T23:21:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.202+0000 7f8e4affd700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-07T23:21:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-07T23:21:00.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.202+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-07T23:21:00.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.886+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50472727 0 0) 0x7f8e4c159670 con 0x7f8e2c0036e0 2023-10-07T23:21:00.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.886+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e38003570 con 0x7f8e2c0036e0 2023-10-07T23:21:00.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.1 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288154119 0 0) 0x7f8e38003570 con 0x7f8e2c0036e0 2023-10-07T23:21:00.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 msgr2=0x7f8e2c0097b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e2c007300 0x7f8e2c0097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c00afd0 con 0x7f8e2c0036e0 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.1 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f8e44001870 con 0x7f8e2c0036e0 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 >> v1:172.21.15.154:6789/0 conn(0x7f8e2c0036e0 legacy=0x7f8e2c005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 reap_dead start 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 0x7f8e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 0x7f8e3800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c06a070 con 0x7f8e38003a60 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3800c250 con 0x7f8e38004810 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3800c410 con 0x7f8e380052e0 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 0x7f8e3800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 0x7f8e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4b7fe700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 msgr2=0x7f8e38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:00.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 0x7f8e38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 msgr2=0x7f8e38003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e38003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4bfff700 1 -- 172.21.15.57:0/2167987857 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38003570 con 0x7f8e380052e0 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4b7fe700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e38003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.890+0000 7f8e4affd700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 0x7f8e38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:00.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e4bfff700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 0x7f8e3800ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c159c10 tx=0x7f8e34008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfdc03c3abbfa29 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:00.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011490 con 0x7f8e380052e0 2023-10-07T23:21:00.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34013720 con 0x7f8e380052e0 2023-10-07T23:21:00.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e4a7fc700 1 -- 172.21.15.57:0/2167987857 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34011ee0 con 0x7f8e380052e0 2023-10-07T23:21:00.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 msgr2=0x7f8e3800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:00.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 0x7f8e3800ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c159c10 tx=0x7f8e34008040 comp rx=0 tx=0).stop 2023-10-07T23:21:00.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 shutdown_connections 2023-10-07T23:21:00.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e380052e0 0x7f8e3800ba60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e38004810 0x7f8e38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 --2- 172.21.15.57:0/2167987857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e38003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 >> 172.21.15.57:0/2167987857 conn(0x7f8e4c06b5d0 msgr2=0x7f8e4c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 shutdown_connections 2023-10-07T23:21:00.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.894+0000 7f8e50f67700 1 -- 172.21.15.57:0/2167987857 wait complete. 2023-10-07T23:21:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 Processor -- start 2023-10-07T23:21:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 -- start start 2023-10-07T23:21:00.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e38004810 0x7f8e4c16b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e380052e0 0x7f8e4c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b0900 con 0x7f8e38003a60 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b0b70 con 0x7f8e380052e0 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e50f67700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b0e50 con 0x7f8e38004810 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42182/0 (socket says 172.21.15.57:42182) 2023-10-07T23:21:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e380052e0 0x7f8e4c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 -- 172.21.15.57:0/1390176556 learned_addr learned my addr 172.21.15.57:0/1390176556 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4affd700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e38004810 0x7f8e4c16b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:00.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 -- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e38004810 msgr2=0x7f8e4c16b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e38004810 0x7f8e4c16b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 -- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e380052e0 msgr2=0x7f8e4c0c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e380052e0 0x7f8e4c0c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 -- 172.21.15.57:0/1390176556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0b1130 con 0x7f8e38003a60 2023-10-07T23:21:00.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.898+0000 7f8e4b7fe700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e44007390 tx=0x7f8e440021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400ae20 con 0x7f8e38003a60 2023-10-07T23:21:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44013600 con 0x7f8e38003a60 2023-10-07T23:21:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c169e40 con 0x7f8e38003a60 2023-10-07T23:21:00.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44013980 con 0x7f8e38003a60 2023-10-07T23:21:00.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c0c6c50 con 0x7f8e38003a60 2023-10-07T23:21:00.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8e4401b3b0 con 0x7f8e38003a60 2023-10-07T23:21:00.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8e440246a0 con 0x7f8e38003a60 2023-10-07T23:21:00.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.902+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c0711c0 con 0x7f8e38003a60 2023-10-07T23:21:00.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.906+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8e4403b6d0 con 0x7f8e38003a60 2023-10-07T23:21:00.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.978+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8e4c0c8ee0 con 0x7f8e38003a60 2023-10-07T23:21:00.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.982+0000 7f8e437fe700 1 -- 172.21.15.57:0/1390176556 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8e4402a070 con 0x7f8e38003a60 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 msgr2=0x7f8e4c169900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e44007390 tx=0x7f8e440021f0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 shutdown_connections 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e380052e0 0x7f8e4c0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e38004810 0x7f8e4c16b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 --2- 172.21.15.57:0/1390176556 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38003a60 0x7f8e4c169900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:00.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 >> 172.21.15.57:0/1390176556 conn(0x7f8e4c06b5d0 msgr2=0x7f8e4c0c1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:00.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 shutdown_connections 2023-10-07T23:21:00.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:00.986+0000 7f8e50f67700 1 -- 172.21.15.57:0/1390176556 wait complete. 2023-10-07T23:21:00.992 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2023-10-07T23:21:01.006 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-07T23:21:01.006 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:21:01.006 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-07T23:21:01.019 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:21:01.019 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-07T23:21:01.032 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:21:01.033 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-07T23:21:01.045 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2023-10-07T23:21:01.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-07T23:21:01.086 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:01.082+0000 7fb40ad3b000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:21:01.090 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:01.085+0000 7f5c7800a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-07T23:21:01.101 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:21:01.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-07T23:21:01.117 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:21:01.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-07T23:21:01.173 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:21:01.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-07T23:21:01.188 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-10-07T23:21:01.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-07T23:21:01.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 4dedf766-fa5e-4cc1-bef2-7238b3aa9a20 0 2023-10-07T23:21:01.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 Processor -- start 2023-10-07T23:21:01.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 -- start start 2023-10-07T23:21:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f081c0681b0 0x7f081c068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c159010 con 0x7f081c0cb110 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.297+0000 7f0821edb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c068b70 con 0x7f081c0681b0 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c155780 con 0x7f081c069130 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0813fff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f081c0cb110 0x7f081c159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48278/0 (socket says 172.21.15.154:48278) 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49796/0 (socket says 172.21.15.154:49796) 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0813fff700 1 -- 172.21.15.154:0/4165390950 learned_addr learned my addr 172.21.15.154:0/4165390950 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f08216da700 1 --2- 172.21.15.154:0/4165390950 >> v2:172.21.15.57:3300/0 conn(0x7f081c0681b0 0x7f081c068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 -- 172.21.15.154:0/4165390950 >> v2:172.21.15.57:3300/0 conn(0x7f081c0681b0 msgr2=0x7f081c068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 --2- 172.21.15.154:0/4165390950 >> v2:172.21.15.57:3300/0 conn(0x7f081c0681b0 0x7f081c068630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f08137fe700 1 -- 172.21.15.154:0/4165390950 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4184922202 0 0) 0x7f081c159010 con 0x7f081c0cb110 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 -- 172.21.15.154:0/4165390950 >> v1:172.21.15.154:6789/0 conn(0x7f081c0cb110 legacy=0x7f081c159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 -- 172.21.15.154:0/4165390950 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0cb880 con 0x7f081c069130 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0820ed9700 1 --2- 172.21.15.154:0/4165390950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08180098e0 tx=0x7f08180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9523f91904e030c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f08137fe700 1 -- 172.21.15.154:0/4165390950 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800b640 con 0x7f081c069130 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f08137fe700 1 -- 172.21.15.154:0/4165390950 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800b7a0 con 0x7f081c069130 2023-10-07T23:21:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f08137fe700 1 -- 172.21.15.154:0/4165390950 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800bb40 con 0x7f081c069130 2023-10-07T23:21:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- 172.21.15.154:0/4165390950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 msgr2=0x7f081c0ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 --2- 172.21.15.154:0/4165390950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08180098e0 tx=0x7f08180091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- 172.21.15.154:0/4165390950 shutdown_connections 2023-10-07T23:21:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 --2- 172.21.15.154:0/4165390950 >> v2:172.21.15.57:3300/0 conn(0x7f081c0681b0 0x7f081c068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 --2- 172.21.15.154:0/4165390950 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c069130 0x7f081c0ca8d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- 172.21.15.154:0/4165390950 >> 172.21.15.154:0/4165390950 conn(0x7f081c0c9740 msgr2=0x7f081c06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- 172.21.15.154:0/4165390950 shutdown_connections 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- 172.21.15.154:0/4165390950 wait complete. 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 Processor -- start 2023-10-07T23:21:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.301+0000 7f0821edb700 1 -- start start 2023-10-07T23:21:01.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f081c069130 0x7f081c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f081c0cb110 0x7f081c0c1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0a6fe0 con 0x7f081c069130 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c1576c0 con 0x7f081c0cb110 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0821edb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c156d90 con 0x7f081c0681b0 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49810/0 (socket says 172.21.15.154:49810) 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 -- 172.21.15.154:0/3240436245 learned_addr learned my addr 172.21.15.154:0/3240436245 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f08216da700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f081c0cb110 0x7f081c0c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0813fff700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f081c069130 0x7f081c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 -- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f081c0cb110 msgr2=0x7f081c0c1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f081c0cb110 0x7f081c0c1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 -- 172.21.15.154:0/3240436245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f081c069130 msgr2=0x7f081c153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f081c069130 0x7f081c153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 -- 172.21.15.154:0/3240436245 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009420 con 0x7f081c0681b0 2023-10-07T23:21:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.305+0000 7f0820ed9700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f081c06f1a0 tx=0x7f0818017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800bb20 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818000c90 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c157070 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818011630 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c16a210 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f08180119a0 con 0x7f081c0681b0 2023-10-07T23:21:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f081802c8d0 con 0x7f081c0681b0 2023-10-07T23:21:01.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.309+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c0711c0 con 0x7f081c0681b0 2023-10-07T23:21:01.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.313+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0818031060 con 0x7f081c0681b0 2023-10-07T23:21:01.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.385+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4dedf766-fa5e-4cc1-bef2-7238b3aa9a20", "id": 0} v 0) v1 -- 0x7f081c0caef0 con 0x7f081c0681b0 2023-10-07T23:21:01.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.397+0000 7f0810ff9700 1 -- 172.21.15.154:0/3240436245 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4dedf766-fa5e-4cc1-bef2-7238b3aa9a20", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0818015020 con 0x7f081c0681b0 2023-10-07T23:21:01.403 INFO:teuthology.orchestra.run.smithi154.stdout:0 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 msgr2=0x7f081c16a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f081c06f1a0 tx=0x7f0818017000 comp rx=0 tx=0).stop 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 shutdown_connections 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f081c0cb110 0x7f081c0c1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f081c069130 0x7f081c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 --2- 172.21.15.154:0/3240436245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f081c0681b0 0x7f081c16a4d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 >> 172.21.15.154:0/3240436245 conn(0x7f081c0c9740 msgr2=0x7f081c1549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 shutdown_connections 2023-10-07T23:21:01.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.401+0000 7f0821edb700 1 -- 172.21.15.154:0/3240436245 wait complete. 2023-10-07T23:21:01.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new ea9dba34-6392-410e-8ae1-e990532b8721 1 2023-10-07T23:21:01.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.517+0000 7f2093e6b700 1 Processor -- start 2023-10-07T23:21:01.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.517+0000 7f2093e6b700 1 -- start start 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 0x7f208c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c15a9b0 con 0x7f208c156b60 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c1579a0 con 0x7f208c0acd00 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c157c80 con 0x7f208c0ac8b0 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49824/0 (socket says 172.21.15.154:49824) 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 -- 172.21.15.154:0/496985498 learned_addr learned my addr 172.21.15.154:0/496985498 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2092408700 1 --2- 172.21.15.154:0/496985498 >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 0x7f208c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 -- 172.21.15.154:0/496985498 >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 msgr2=0x7f208c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 --2- 172.21.15.154:0/496985498 >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 0x7f208c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 -- 172.21.15.154:0/496985498 >> v1:172.21.15.57:6789/0 conn(0x7f208c156b60 legacy=0x7f208c068140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 -- 172.21.15.154:0/496985498 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c157f60 con 0x7f208c0ac8b0 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2092408700 1 --2- 172.21.15.154:0/496985498 >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 0x7f208c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2091406700 1 --2- 172.21.15.154:0/496985498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f207c0099b0 tx=0x7f207c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d29d6d9b70abd1e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2090c05700 1 -- 172.21.15.154:0/496985498 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00b860 con 0x7f208c0ac8b0 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2090c05700 1 -- 172.21.15.154:0/496985498 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c00b9c0 con 0x7f208c0ac8b0 2023-10-07T23:21:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2090c05700 1 -- 172.21.15.154:0/496985498 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c00bd60 con 0x7f208c0ac8b0 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- 172.21.15.154:0/496985498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 msgr2=0x7f208c156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 --2- 172.21.15.154:0/496985498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f207c0099b0 tx=0x7f207c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- 172.21.15.154:0/496985498 shutdown_connections 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 --2- 172.21.15.154:0/496985498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c156f30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 --2- 172.21.15.154:0/496985498 >> v2:172.21.15.57:3300/0 conn(0x7f208c0acd00 0x7f208c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- 172.21.15.154:0/496985498 >> 172.21.15.154:0/496985498 conn(0x7f208c06b720 msgr2=0x7f208c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- 172.21.15.154:0/496985498 shutdown_connections 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.521+0000 7f2093e6b700 1 -- 172.21.15.154:0/496985498 wait complete. 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 Processor -- start 2023-10-07T23:21:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- start start 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c0c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f208c0acd00 0x7f208c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0c2db0 con 0x7f208c0acd00 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c15d2f0 con 0x7f208c156b60 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c15e110 con 0x7f208c0ac8b0 2023-10-07T23:21:01.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33608/0 (socket says 172.21.15.154:33608) 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 -- 172.21.15.154:0/1164153338 learned_addr learned my addr 172.21.15.154:0/1164153338 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2091406700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f208c0acd00 0x7f208c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2091c07700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c0c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 -- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 msgr2=0x7f208c0c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c0c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 -- 172.21.15.154:0/1164153338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f208c0acd00 msgr2=0x7f208c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f208c0acd00 0x7f208c15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 -- 172.21.15.154:0/1164153338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0094f0 con 0x7f208c156b60 2023-10-07T23:21:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2092408700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f208400d150 tx=0x7f2084009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208400dc30 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208400f040 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c15d500 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084015400 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c15dac0 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f20840157b0 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.525+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f208401e9d0 con 0x7f208c156b60 2023-10-07T23:21:01.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.529+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c067940 con 0x7f208c156b60 2023-10-07T23:21:01.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.533+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f208c067940 con 0x7f208c156b60 2023-10-07T23:21:01.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.609+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea9dba34-6392-410e-8ae1-e990532b8721", "id": 1} v 0) v1 -- 0x7f208c0647b0 con 0x7f208c156b60 2023-10-07T23:21:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2081ffb700 1 -- 172.21.15.154:0/1164153338 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea9dba34-6392-410e-8ae1-e990532b8721", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f208402d0b0 con 0x7f208c156b60 2023-10-07T23:21:01.628 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 msgr2=0x7f208c15eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f208400d150 tx=0x7f2084009a50 comp rx=0 tx=0).stop 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 shutdown_connections 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f208c156b60 0x7f208c15eab0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f208c0acd00 0x7f208c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 --2- 172.21.15.154:0/1164153338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f208c0ac8b0 0x7f208c0c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 >> 172.21.15.154:0/1164153338 conn(0x7f208c06b720 msgr2=0x7f208c068060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 shutdown_connections 2023-10-07T23:21:01.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.625+0000 7f2093e6b700 1 -- 172.21.15.154:0/1164153338 wait complete. 2023-10-07T23:21:01.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new b21b4335-8486-43d1-90ab-bc9f750452fe 2 2023-10-07T23:21:01.725 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:01.721+0000 7f5c7800a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:21:01.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 Processor -- start 2023-10-07T23:21:01.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 -- start start 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0c9150 con 0x7f993c0afaf0 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0ae0b0 con 0x7f993c0ad5b0 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f9942100700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0c8920 con 0x7f993c156800 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f99418ff700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f993c0ad5b0 0x7f993c0ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48282/0 (socket says 172.21.15.154:48282) 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f99418ff700 1 -- 172.21.15.154:0/4047320617 learned_addr learned my addr 172.21.15.154:0/4047320617 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.737+0000 7f99408fd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f99408fd700 1 -- 172.21.15.154:0/4047320617 >> v1:172.21.15.154:6789/0 conn(0x7f993c0ad5b0 legacy=0x7f993c0ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f99408fd700 1 -- 172.21.15.154:0/4047320617 >> v1:172.21.15.57:6789/0 conn(0x7f993c0afaf0 legacy=0x7f993c0afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9933fff700 1 -- 172.21.15.154:0/4047320617 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252045925 0 0) 0x7f993c0ae0b0 con 0x7f993c0ad5b0 2023-10-07T23:21:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f99408fd700 1 -- 172.21.15.154:0/4047320617 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0ae300 con 0x7f993c156800 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f99408fd700 1 --2- 172.21.15.154:0/4047320617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c156c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f992c009910 tx=0x7f992c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5c3ad338c3fee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9933fff700 1 -- 172.21.15.154:0/4047320617 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00b860 con 0x7f993c156800 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9933fff700 1 -- 172.21.15.154:0/4047320617 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c00b9c0 con 0x7f993c156800 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9933fff700 1 -- 172.21.15.154:0/4047320617 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00bd60 con 0x7f993c156800 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- 172.21.15.154:0/4047320617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 msgr2=0x7f993c156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 --2- 172.21.15.154:0/4047320617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c156c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f992c009910 tx=0x7f992c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- 172.21.15.154:0/4047320617 shutdown_connections 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 --2- 172.21.15.154:0/4047320617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c156c30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- 172.21.15.154:0/4047320617 >> 172.21.15.154:0/4047320617 conn(0x7f993c0692f0 msgr2=0x7f993c0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- 172.21.15.154:0/4047320617 shutdown_connections 2023-10-07T23:21:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- 172.21.15.154:0/4047320617 wait complete. 2023-10-07T23:21:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 Processor -- start 2023-10-07T23:21:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.741+0000 7f9942100700 1 -- start start 2023-10-07T23:21:01.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 0x7f993c146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0c22a0 con 0x7f993c0ad5b0 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c159ab0 con 0x7f993c0afaf0 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c159d90 con 0x7f993c156800 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99418ff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33624/0 (socket says 172.21.15.154:33624) 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 -- 172.21.15.154:0/1108381635 learned_addr learned my addr 172.21.15.154:0/1108381635 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99410fe700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 0x7f993c146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 -- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 msgr2=0x7f993c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 -- 172.21.15.154:0/1108381635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 msgr2=0x7f993c146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 0x7f993c146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 -- 172.21.15.154:0/1108381635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c009420 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99410fe700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 0x7f993c146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99408fd700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f993c06c0c0 tx=0x7f992c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c01f070 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c002f70 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c15a070 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c025420 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c158d10 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f992c023070 con 0x7f993c0afaf0 2023-10-07T23:21:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f992c01a420 con 0x7f993c0afaf0 2023-10-07T23:21:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.745+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c063750 con 0x7f993c0afaf0 2023-10-07T23:21:01.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.749+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f993c063750 con 0x7f993c0afaf0 2023-10-07T23:21:01.755 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:01.750+0000 7fb40ad3b000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-07T23:21:01.784 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:01.781+0000 7f5c7800a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:21:01.813 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:01.806+0000 7fb40ad3b000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-07T23:21:01.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.829+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b21b4335-8486-43d1-90ab-bc9f750452fe", "id": 2} v 0) v1 -- 0x7f993c0c2410 con 0x7f993c0afaf0 2023-10-07T23:21:01.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.841+0000 7f99317fa700 1 -- 172.21.15.154:0/1108381635 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b21b4335-8486-43d1-90ab-bc9f750452fe", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f992c02a070 con 0x7f993c0afaf0 2023-10-07T23:21:01.846 INFO:teuthology.orchestra.run.smithi154.stdout:2 2023-10-07T23:21:01.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 msgr2=0x7f993c0b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f993c06c0c0 tx=0x7f992c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 shutdown_connections 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f993c156800 0x7f993c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c0afaf0 0x7f993c0b2970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 --2- 172.21.15.154:0/1108381635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f993c0ad5b0 0x7f993c146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 >> 172.21.15.154:0/1108381635 conn(0x7f993c0692f0 msgr2=0x7f993c0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 shutdown_connections 2023-10-07T23:21:01.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.845+0000 7f9942100700 1 -- 172.21.15.154:0/1108381635 wait complete. 2023-10-07T23:21:01.864 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3 3 2023-10-07T23:21:01.898 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:01.893+0000 7f5c7800a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:21:01.931 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:01.926+0000 7fb40ad3b000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:21:01.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 Processor -- start 2023-10-07T23:21:01.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 -- start start 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3bc156f30 0x7fe3bc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe3bc0c9670 0x7fe3bc0c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc06d470 con 0x7fe3bc157300 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc06c8a0 con 0x7fe3bc0c9670 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c4350700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc06cb40 con 0x7fe3bc156f30 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3bc156f30 0x7fe3bc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe3bc156f30 0x7fe3bc06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33630/0 (socket says 172.21.15.154:33630) 2023-10-07T23:21:01.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 -- 172.21.15.154:0/459303411 learned_addr learned my addr 172.21.15.154:0/459303411 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c18eb700 1 --1- 172.21.15.154:0/459303411 >> v1:172.21.15.154:6789/0 conn(0x7fe3bc157300 0x7fe3bc0bfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48292/0 (socket says 172.21.15.154:48292) 2023-10-07T23:21:01.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c28ed700 1 --2- 172.21.15.154:0/459303411 >> v2:172.21.15.57:3300/0 conn(0x7fe3bc0c9670 0x7fe3bc0c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 -- 172.21.15.154:0/459303411 >> v2:172.21.15.57:3300/0 conn(0x7fe3bc0c9670 msgr2=0x7fe3bc0c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:01.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 --2- 172.21.15.154:0/459303411 >> v2:172.21.15.57:3300/0 conn(0x7fe3bc0c9670 0x7fe3bc0c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 -- 172.21.15.154:0/459303411 >> v1:172.21.15.154:6789/0 conn(0x7fe3bc157300 legacy=0x7fe3bc0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 -- 172.21.15.154:0/459303411 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc06ce20 con 0x7fe3bc156f30 2023-10-07T23:21:01.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c20ec700 1 --2- 172.21.15.154:0/459303411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc156f30 0x7fe3bc06d900 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc069f70 tx=0x7fe3ac009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e2bdca10ab9ea7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c10ea700 1 -- 172.21.15.154:0/459303411 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac009e70 con 0x7fe3bc156f30 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c10ea700 1 -- 172.21.15.154:0/459303411 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00b860 con 0x7fe3bc156f30 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.969+0000 7fe3c10ea700 1 -- 172.21.15.154:0/459303411 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00bc00 con 0x7fe3bc156f30 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- 172.21.15.154:0/459303411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc156f30 msgr2=0x7fe3bc06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- 172.21.15.154:0/459303411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc156f30 0x7fe3bc06d900 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc069f70 tx=0x7fe3ac009250 comp rx=0 tx=0).stop 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- 172.21.15.154:0/459303411 shutdown_connections 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- 172.21.15.154:0/459303411 >> v2:172.21.15.57:3300/0 conn(0x7fe3bc0c9670 0x7fe3bc0c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- 172.21.15.154:0/459303411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc156f30 0x7fe3bc06d900 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- 172.21.15.154:0/459303411 >> 172.21.15.154:0/459303411 conn(0x7fe3bc0ccf10 msgr2=0x7fe3bc0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- 172.21.15.154:0/459303411 shutdown_connections 2023-10-07T23:21:01.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- 172.21.15.154:0/459303411 wait complete. 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 Processor -- start 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- start start 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 0x7fe3bc154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3bc156f30 0x7fe3bc1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc167210 con 0x7fe3bc156f30 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0afa60 con 0x7fe3bc0c9670 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c4350700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0c5f50 con 0x7fe3bc157300 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c20ec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 0x7fe3bc154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49842/0 (socket says 172.21.15.154:49842) 2023-10-07T23:21:01.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 -- 172.21.15.154:0/639691457 learned_addr learned my addr 172.21.15.154:0/639691457 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:01.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c18eb700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3bc156f30 0x7fe3bc1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 -- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 msgr2=0x7fe3bc154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 0x7fe3bc154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 -- 172.21.15.154:0/639691457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3bc156f30 msgr2=0x7fe3bc1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3bc156f30 0x7fe3bc1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c28ed700 1 -- 172.21.15.154:0/639691457 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0094f0 con 0x7fe3bc157300 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.973+0000 7fe3c20ec700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 0x7fe3bc154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3c28ed700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400ea80 tx=0x7fe3b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:01.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400eec0 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc154570 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b400c7f0 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400cbb0 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc15a520 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe3b40136f0 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fe3b401c4f0 con 0x7fe3bc157300 2023-10-07T23:21:01.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.977+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc15a860 con 0x7fe3bc157300 2023-10-07T23:21:01.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:01.981+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe3b4021060 con 0x7fe3bc157300 2023-10-07T23:21:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.053+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3", "id": 3} v 0) v1 -- 0x7fe3bc0647b0 con 0x7fe3bc157300 2023-10-07T23:21:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3ba7fc700 1 -- 172.21.15.154:0/639691457 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe3b402a020 con 0x7fe3bc157300 2023-10-07T23:21:02.069 INFO:teuthology.orchestra.run.smithi154.stdout:3 2023-10-07T23:21:02.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 msgr2=0x7fe3bc159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400ea80 tx=0x7fe3b40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 shutdown_connections 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3bc157300 0x7fe3bc159c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3bc156f30 0x7fe3bc1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 --2- 172.21.15.154:0/639691457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3bc0c9670 0x7fe3bc154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 >> 172.21.15.154:0/639691457 conn(0x7fe3bc0ccf10 msgr2=0x7fe3bc155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 shutdown_connections 2023-10-07T23:21:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.065+0000 7fe3c4350700 1 -- 172.21.15.154:0/639691457 wait complete. 2023-10-07T23:21:02.083 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new d167c366-16e6-47d5-9be7-2700c4f4e71a 4 2023-10-07T23:21:02.125 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.121+0000 7f5c7800a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:21:02.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 Processor -- start 2023-10-07T23:21:02.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 -- start start 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6a04158ce0 0x7f6a0415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a0415d080 0x7f6a040c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0406c800 con 0x7f6a041590b0 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0406cdc0 con 0x7f6a04158ce0 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a040ae850 con 0x7f6a0415d080 2023-10-07T23:21:02.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a01d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a0415d080 0x7f6a040c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.176 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f5c7800a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:21:02.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a01d9b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a0415d080 0x7f6a040c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33650/0 (socket says 172.21.15.154:33650) 2023-10-07T23:21:02.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a01d9b700 1 -- 172.21.15.154:0/1766931888 learned_addr learned my addr 172.21.15.154:0/1766931888 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.169+0000 7f6a0259c700 1 --2- 172.21.15.154:0/1766931888 >> v2:172.21.15.57:3300/0 conn(0x7f6a04158ce0 0x7f6a0415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a01d9b700 1 -- 172.21.15.154:0/1766931888 >> v2:172.21.15.57:3300/0 conn(0x7f6a04158ce0 msgr2=0x7f6a0415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a01d9b700 1 --2- 172.21.15.154:0/1766931888 >> v2:172.21.15.57:3300/0 conn(0x7f6a04158ce0 0x7f6a0415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a01d9b700 1 -- 172.21.15.154:0/1766931888 >> v1:172.21.15.57:6789/0 conn(0x7f6a041590b0 legacy=0x7f6a0406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a01d9b700 1 -- 172.21.15.154:0/1766931888 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0415d600 con 0x7f6a0415d080 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a01d9b700 1 --2- 172.21.15.154:0/1766931888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a040c93c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a04069af0 tx=0x7f69f8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3941d7013fc25a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0159a700 1 -- 172.21.15.154:0/1766931888 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800b860 con 0x7f6a0415d080 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0159a700 1 -- 172.21.15.154:0/1766931888 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f800b9c0 con 0x7f6a0415d080 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0159a700 1 -- 172.21.15.154:0/1766931888 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800bd60 con 0x7f6a0415d080 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 -- 172.21.15.154:0/1766931888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 msgr2=0x7f6a040c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 --2- 172.21.15.154:0/1766931888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a040c93c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a04069af0 tx=0x7f69f8009250 comp rx=0 tx=0).stop 2023-10-07T23:21:02.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 -- 172.21.15.154:0/1766931888 shutdown_connections 2023-10-07T23:21:02.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 --2- 172.21.15.154:0/1766931888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a040c93c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 --2- 172.21.15.154:0/1766931888 >> v2:172.21.15.57:3300/0 conn(0x7f6a04158ce0 0x7f6a0415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 -- 172.21.15.154:0/1766931888 >> 172.21.15.154:0/1766931888 conn(0x7f6a04067b40 msgr2=0x7f6a040c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 -- 172.21.15.154:0/1766931888 shutdown_connections 2023-10-07T23:21:02.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.173+0000 7f6a0359e700 1 -- 172.21.15.154:0/1766931888 wait complete. 2023-10-07T23:21:02.179 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:02.170+0000 7fb40ad3b000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-07T23:21:02.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 Processor -- start 2023-10-07T23:21:02.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 -- start start 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a041590b0 0x7f6a04157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a041577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b0c60 con 0x7f6a041590b0 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b0f40 con 0x7f6a0415d080 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040b1220 con 0x7f6a04158ce0 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49858/0 (socket says 172.21.15.154:49858) 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 -- 172.21.15.154:0/3435316548 learned_addr learned my addr 172.21.15.154:0/3435316548 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a02d9d700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a041577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a01d9b700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a041590b0 0x7f6a04157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 -- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 msgr2=0x7f6a041577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a041577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 -- 172.21.15.154:0/3435316548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a041590b0 msgr2=0x7f6a04157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a041590b0 0x7f6a04157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 -- 172.21.15.154:0/3435316548 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80094f0 con 0x7f6a04158ce0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.177+0000 7f6a0259c700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009fa0 tx=0x7f69f4009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400ebc0 con 0x7f6a04158ce0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f400ed20 con 0x7f6a04158ce0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a040b1500 con 0x7f6a04158ce0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4014670 con 0x7f6a04158ce0 2023-10-07T23:21:02.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a04159e30 con 0x7f6a04158ce0 2023-10-07T23:21:02.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f69f40149e0 con 0x7f6a04158ce0 2023-10-07T23:21:02.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.181+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f69f4014e20 con 0x7f6a04158ce0 2023-10-07T23:21:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.185+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a040711c0 con 0x7f6a04158ce0 2023-10-07T23:21:02.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.185+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f69f401e5f0 con 0x7f6a04158ce0 2023-10-07T23:21:02.228 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:02.222+0000 7fb40ad3b000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-07T23:21:02.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.261+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d167c366-16e6-47d5-9be7-2700c4f4e71a", "id": 4} v 0) v1 -- 0x7f6a04063030 con 0x7f6a04158ce0 2023-10-07T23:21:02.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f69f27fc700 1 -- 172.21.15.154:0/3435316548 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d167c366-16e6-47d5-9be7-2700c4f4e71a", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f69f402d0b0 con 0x7f6a04158ce0 2023-10-07T23:21:02.276 INFO:teuthology.orchestra.run.smithi154.stdout:4 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 msgr2=0x7f6a04143c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009fa0 tx=0x7f69f4009880 comp rx=0 tx=0).stop 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 shutdown_connections 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a0415d080 0x7f6a041577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6a041590b0 0x7f6a04157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 --2- 172.21.15.154:0/3435316548 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6a04158ce0 0x7f6a04143c30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 >> 172.21.15.154:0/3435316548 conn(0x7f6a04067b40 msgr2=0x7f6a040cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 shutdown_connections 2023-10-07T23:21:02.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.273+0000 7f6a0359e700 1 -- 172.21.15.154:0/3435316548 wait complete. 2023-10-07T23:21:02.289 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 012ff59d-c3fd-48aa-9b9d-c4cab134c7aa 5 2023-10-07T23:21:02.354 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.349+0000 7f5c7800a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:21:02.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.389+0000 7f65502c3700 1 Processor -- start 2023-10-07T23:21:02.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.389+0000 7f65502c3700 1 -- start start 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65480c73f0 0x7f65480693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6548069900 con 0x7f65481554f0 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480c7870 con 0x7f6548068a10 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480aee70 con 0x7f65480c73f0 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65480c73f0 0x7f65480693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65480c73f0 0x7f65480693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33664/0 (socket says 172.21.15.154:33664) 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 -- 172.21.15.154:0/2320748405 learned_addr learned my addr 172.21.15.154:0/2320748405 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 -- 172.21.15.154:0/2320748405 >> v1:172.21.15.154:6789/0 conn(0x7f6548068a10 legacy=0x7f65480ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 -- 172.21.15.154:0/2320748405 >> v1:172.21.15.57:6789/0 conn(0x7f65481554f0 legacy=0x7f6548068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 -- 172.21.15.154:0/2320748405 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480acb70 con 0x7f65480c73f0 2023-10-07T23:21:02.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654e860700 1 --2- 172.21.15.154:0/2320748405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65480c73f0 0x7f65480693c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f654000c0d0 tx=0x7f654000a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425be16932e92e04 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654d05d700 1 -- 172.21.15.154:0/2320748405 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000f070 con 0x7f65480c73f0 2023-10-07T23:21:02.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654d05d700 1 -- 172.21.15.154:0/2320748405 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000a2e0 con 0x7f65480c73f0 2023-10-07T23:21:02.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f654d05d700 1 -- 172.21.15.154:0/2320748405 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000cdf0 con 0x7f65480c73f0 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- 172.21.15.154:0/2320748405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65480c73f0 msgr2=0x7f65480693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 --2- 172.21.15.154:0/2320748405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65480c73f0 0x7f65480693c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f654000c0d0 tx=0x7f654000a950 comp rx=0 tx=0).stop 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- 172.21.15.154:0/2320748405 shutdown_connections 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 --2- 172.21.15.154:0/2320748405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65480c73f0 0x7f65480693c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- 172.21.15.154:0/2320748405 >> 172.21.15.154:0/2320748405 conn(0x7f65480ccf10 msgr2=0x7f65480c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- 172.21.15.154:0/2320748405 shutdown_connections 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- 172.21.15.154:0/2320748405 wait complete. 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 Processor -- start 2023-10-07T23:21:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.393+0000 7f65502c3700 1 -- start start 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 0x7f65480aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65481554f0 0x7f65480c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654816a260 con 0x7f65480c73f0 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548156530 con 0x7f6548068a10 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548156810 con 0x7f65481554f0 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e860700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65481554f0 0x7f65480c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33674/0 (socket says 172.21.15.154:33674) 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 -- 172.21.15.154:0/1680242365 learned_addr learned my addr 172.21.15.154:0/1680242365 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654d85e700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 0x7f65480aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 -- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65481554f0 msgr2=0x7f65480c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65481554f0 0x7f65480c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 -- 172.21.15.154:0/1680242365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 msgr2=0x7f65480aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 0x7f65480aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 -- 172.21.15.154:0/1680242365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654000abf0 con 0x7f6548068a10 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654d85e700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 0x7f65480aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f654e05f700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f654400b5e0 tx=0x7f6544009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400f510 con 0x7f6548068a10 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400f670 con 0x7f6548068a10 2023-10-07T23:21:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6548156af0 con 0x7f6548068a10 2023-10-07T23:21:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400fa30 con 0x7f6548068a10 2023-10-07T23:21:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6548156dd0 con 0x7f6548068a10 2023-10-07T23:21:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f654401a400 con 0x7f6548068a10 2023-10-07T23:21:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f654400fe20 con 0x7f6548068a10 2023-10-07T23:21:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.397+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654816a3a0 con 0x7f6548068a10 2023-10-07T23:21:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.401+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f654816a3a0 con 0x7f6548068a10 2023-10-07T23:21:02.414 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.409+0000 7f5c7800a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:21:02.421 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:02.414+0000 7fb40ad3b000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-07T23:21:02.475 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.469+0000 7f5c7800a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:21:02.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.477+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "012ff59d-c3fd-48aa-9b9d-c4cab134c7aa", "id": 5} v 0) v1 -- 0x7f65480647b0 con 0x7f6548068a10 2023-10-07T23:21:02.489 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:02.482+0000 7fb40ad3b000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-07T23:21:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.489+0000 7f653e7fc700 1 -- 172.21.15.154:0/1680242365 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "012ff59d-c3fd-48aa-9b9d-c4cab134c7aa", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f65440290b0 con 0x7f6548068a10 2023-10-07T23:21:02.495 INFO:teuthology.orchestra.run.smithi154.stdout:5 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 msgr2=0x7f6548155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f654400b5e0 tx=0x7f6544009120 comp rx=0 tx=0).stop 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 shutdown_connections 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f65481554f0 0x7f65480c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65480c73f0 0x7f65480aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 --2- 172.21.15.154:0/1680242365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6548068a10 0x7f6548155d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 >> 172.21.15.154:0/1680242365 conn(0x7f65480ccf10 msgr2=0x7f6548068300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 shutdown_connections 2023-10-07T23:21:02.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.493+0000 7f65502c3700 1 -- 172.21.15.154:0/1680242365 wait complete. 2023-10-07T23:21:02.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 18cd0831-a7c6-404a-af7b-ed249e03fcfd 6 2023-10-07T23:21:02.557 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:02.550+0000 7fb40ad3b000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-07T23:21:02.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 Processor -- start 2023-10-07T23:21:02.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 -- start start 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34fc06dad0 0x7f34fc157c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f34fc0c93c0 0x7f34fc0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc15b800 con 0x7f34fc06dea0 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc15afe0 con 0x7f34fc0c93c0 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3502c00700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0c9d40 con 0x7f34fc06dad0 2023-10-07T23:21:02.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34fc06dad0 0x7f34fc157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34fc06dad0 0x7f34fc157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:33688/0 (socket says 172.21.15.154:33688) 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 -- 172.21.15.154:0/1907478280 learned_addr learned my addr 172.21.15.154:0/1907478280 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f35023ff700 1 --2- 172.21.15.154:0/1907478280 >> v2:172.21.15.57:3300/0 conn(0x7f34fc0c93c0 0x7f34fc0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 -- 172.21.15.154:0/1907478280 >> v2:172.21.15.57:3300/0 conn(0x7f34fc0c93c0 msgr2=0x7f34fc0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 --2- 172.21.15.154:0/1907478280 >> v2:172.21.15.57:3300/0 conn(0x7f34fc0c93c0 0x7f34fc0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 -- 172.21.15.154:0/1907478280 >> v1:172.21.15.154:6789/0 conn(0x7f34fc06dea0 legacy=0x7f34fc15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 -- 172.21.15.154:0/1907478280 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc15b240 con 0x7f34fc06dad0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3501bfe700 1 --2- 172.21.15.154:0/1907478280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc06dad0 0x7f34fc157c70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06a3c0 tx=0x7f34f0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee01e718c5d21cda server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3500bfc700 1 -- 172.21.15.154:0/1907478280 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000b860 con 0x7f34fc06dad0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3500bfc700 1 -- 172.21.15.154:0/1907478280 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000b9c0 con 0x7f34fc06dad0 2023-10-07T23:21:02.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.605+0000 7f3500bfc700 1 -- 172.21.15.154:0/1907478280 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000bd60 con 0x7f34fc06dad0 2023-10-07T23:21:02.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- 172.21.15.154:0/1907478280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc06dad0 msgr2=0x7f34fc157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- 172.21.15.154:0/1907478280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc06dad0 0x7f34fc157c70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06a3c0 tx=0x7f34f0009250 comp rx=0 tx=0).stop 2023-10-07T23:21:02.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- 172.21.15.154:0/1907478280 shutdown_connections 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- 172.21.15.154:0/1907478280 >> v2:172.21.15.57:3300/0 conn(0x7f34fc0c93c0 0x7f34fc0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- 172.21.15.154:0/1907478280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc06dad0 0x7f34fc157c70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- 172.21.15.154:0/1907478280 >> 172.21.15.154:0/1907478280 conn(0x7f34fc067b40 msgr2=0x7f34fc0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- 172.21.15.154:0/1907478280 shutdown_connections 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- 172.21.15.154:0/1907478280 wait complete. 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 Processor -- start 2023-10-07T23:21:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- start start 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34fc06dea0 0x7f34fc1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 0x7f34fc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc156000 con 0x7f34fc06dea0 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1553f0 con 0x7f34fc0c93c0 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3502c00700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1556d0 con 0x7f34fc06dad0 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.609+0000 7f3501bfe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49874/0 (socket says 172.21.15.154:49874) 2023-10-07T23:21:02.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f35023ff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 0x7f34fc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 -- 172.21.15.154:0/155932737 learned_addr learned my addr 172.21.15.154:0/155932737 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f35013fd700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34fc06dea0 0x7f34fc1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 -- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 msgr2=0x7f34fc0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 0x7f34fc0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 -- 172.21.15.154:0/155932737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34fc06dea0 msgr2=0x7f34fc1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34fc06dea0 0x7f34fc1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 -- 172.21.15.154:0/155932737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00094f0 con 0x7f34fc06dad0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f35023ff700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 0x7f34fc0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3501bfe700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0c9e80 tx=0x7f34f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0024440 con 0x7f34fc06dad0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f00245a0 con 0x7f34fc06dad0 2023-10-07T23:21:02.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc1559b0 con 0x7f34fc06dad0 2023-10-07T23:21:02.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0024960 con 0x7f34fc06dad0 2023-10-07T23:21:02.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc14f650 con 0x7f34fc06dad0 2023-10-07T23:21:02.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f34f0024d50 con 0x7f34fc06dad0 2023-10-07T23:21:02.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f34f00239e0 con 0x7f34fc06dad0 2023-10-07T23:21:02.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.613+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc0711c0 con 0x7f34fc06dad0 2023-10-07T23:21:02.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.617+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f34f0015020 con 0x7f34fc06dad0 2023-10-07T23:21:02.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.721+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "18cd0831-a7c6-404a-af7b-ed249e03fcfd", "id": 6} v 0) v1 -- 0x7f34fc062ff0 con 0x7f34fc06dad0 2023-10-07T23:21:02.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.725+0000 7f34edffb700 1 -- 172.21.15.154:0/155932737 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "18cd0831-a7c6-404a-af7b-ed249e03fcfd", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f34f001a880 con 0x7f34fc06dad0 2023-10-07T23:21:02.731 INFO:teuthology.orchestra.run.smithi154.stdout:6 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 msgr2=0x7f34fc14f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0c9e80 tx=0x7f34f0017000 comp rx=0 tx=0).stop 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 shutdown_connections 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34fc0c93c0 0x7f34fc0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f34fc06dea0 0x7f34fc1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 --2- 172.21.15.154:0/155932737 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f34fc06dad0 0x7f34fc14f110 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 >> 172.21.15.154:0/155932737 conn(0x7f34fc067b40 msgr2=0x7f34fc159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 shutdown_connections 2023-10-07T23:21:02.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.729+0000 7f3502c00700 1 -- 172.21.15.154:0/155932737 wait complete. 2023-10-07T23:21:02.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 87479800-b391-4784-8fed-c0278af5f8dc 7 2023-10-07T23:21:02.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.841+0000 7f31e9879700 1 Processor -- start 2023-10-07T23:21:02.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.841+0000 7f31e9879700 1 -- start start 2023-10-07T23:21:02.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31e4069c20 0x7f31e415a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f31e4155b60 0x7f31e40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e4155fa0 con 0x7f31e4155790 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e406a070 con 0x7f31e4155b60 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e4159ca0 con 0x7f31e4069c20 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31e4069c20 0x7f31e415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f31e4069c20 0x7f31e415a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53136/0 (socket says 172.21.15.154:53136) 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 -- 172.21.15.154:0/984551167 learned_addr learned my addr 172.21.15.154:0/984551167 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9078700 1 --2- 172.21.15.154:0/984551167 >> v2:172.21.15.57:3300/0 conn(0x7f31e4155b60 0x7f31e40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 -- 172.21.15.154:0/984551167 >> v2:172.21.15.57:3300/0 conn(0x7f31e4155b60 msgr2=0x7f31e40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 --2- 172.21.15.154:0/984551167 >> v2:172.21.15.57:3300/0 conn(0x7f31e4155b60 0x7f31e40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 -- 172.21.15.154:0/984551167 >> v1:172.21.15.57:6789/0 conn(0x7f31e4155790 legacy=0x7f31e40c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 -- 172.21.15.154:0/984551167 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4159670 con 0x7f31e4069c20 2023-10-07T23:21:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e3fff700 1 --2- 172.21.15.154:0/984551167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4069c20 0x7f31e415a130 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31e4069200 tx=0x7f31d000b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a67886a5ce6389 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e37fe700 1 -- 172.21.15.154:0/984551167 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d000dc50 con 0x7f31e4069c20 2023-10-07T23:21:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e37fe700 1 -- 172.21.15.154:0/984551167 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d000ddb0 con 0x7f31e4069c20 2023-10-07T23:21:02.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e37fe700 1 -- 172.21.15.154:0/984551167 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00196d0 con 0x7f31e4069c20 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- 172.21.15.154:0/984551167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4069c20 msgr2=0x7f31e415a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 --2- 172.21.15.154:0/984551167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4069c20 0x7f31e415a130 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31e4069200 tx=0x7f31d000b4f0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- 172.21.15.154:0/984551167 shutdown_connections 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 --2- 172.21.15.154:0/984551167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4069c20 0x7f31e415a130 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 --2- 172.21.15.154:0/984551167 >> v2:172.21.15.57:3300/0 conn(0x7f31e4155b60 0x7f31e40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- 172.21.15.154:0/984551167 >> 172.21.15.154:0/984551167 conn(0x7f31e406b5d0 msgr2=0x7f31e406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- 172.21.15.154:0/984551167 shutdown_connections 2023-10-07T23:21:02.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.845+0000 7f31e9879700 1 -- 172.21.15.154:0/984551167 wait complete. 2023-10-07T23:21:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 Processor -- start 2023-10-07T23:21:02.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 -- start start 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31e4069c20 0x7f31e40c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 0x7f31e406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 0x7f31e40c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c49b0 con 0x7f31e4069c20 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c43f0 con 0x7f31e4155b60 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c46d0 con 0x7f31e4155790 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9078700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 0x7f31e40c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 0x7f31e406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9078700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 0x7f31e40c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53142/0 (socket says 172.21.15.154:53142) 2023-10-07T23:21:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9078700 1 -- 172.21.15.154:0/2630450655 learned_addr learned my addr 172.21.15.154:0/2630450655 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e8877700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31e4069c20 0x7f31e40c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 -- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 msgr2=0x7f31e40c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 0x7f31e40c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 -- 172.21.15.154:0/2630450655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31e4069c20 msgr2=0x7f31e40c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31e4069c20 0x7f31e40c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 -- 172.21.15.154:0/2630450655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d000b790 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e3fff700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 0x7f31e406d180 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f31e406b3d0 tx=0x7f31d001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0010470 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d0010610 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e406c980 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.849+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00109d0 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.853+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e40c40e0 con 0x7f31e4155790 2023-10-07T23:21:02.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.853+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f31d0022070 con 0x7f31e4155790 2023-10-07T23:21:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.853+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f31d0014070 con 0x7f31e4155790 2023-10-07T23:21:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.853+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e406ce80 con 0x7f31e4155790 2023-10-07T23:21:02.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.853+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f31d0029070 con 0x7f31e4155790 2023-10-07T23:21:02.887 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:02.881+0000 7f5c7800a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:21:02.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.929+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "87479800-b391-4784-8fed-c0278af5f8dc", "id": 7} v 0) v1 -- 0x7f31e40c4810 con 0x7f31e4155790 2023-10-07T23:21:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.937+0000 7f31e0ff9700 1 -- 172.21.15.154:0/2630450655 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87479800-b391-4784-8fed-c0278af5f8dc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f31e40c4810 con 0x7f31e4155790 2023-10-07T23:21:02.943 INFO:teuthology.orchestra.run.smithi154.stdout:7 2023-10-07T23:21:02.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 msgr2=0x7f31e406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:02.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 0x7f31e406d180 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f31e406b3d0 tx=0x7f31d001b000 comp rx=0 tx=0).stop 2023-10-07T23:21:02.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 shutdown_connections 2023-10-07T23:21:02.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e4155b60 0x7f31e40c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f31e4155790 0x7f31e406d180 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 --2- 172.21.15.154:0/2630450655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31e4069c20 0x7f31e40c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 >> 172.21.15.154:0/2630450655 conn(0x7f31e406b5d0 msgr2=0x7f31e415b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 shutdown_connections 2023-10-07T23:21:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-10-07T23:21:02.941+0000 7f31e9879700 1 -- 172.21.15.154:0/2630450655 wait complete. 2023-10-07T23:21:02.955 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-07T23:21:02.955 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-07T23:21:02.957 INFO:tasks.ceph.osd.0:Started 2023-10-07T23:21:02.958 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-07T23:21:02.958 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-07T23:21:02.960 INFO:tasks.ceph.osd.1:Started 2023-10-07T23:21:02.960 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-07T23:21:02.961 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-07T23:21:02.963 INFO:tasks.ceph.osd.2:Started 2023-10-07T23:21:02.963 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-07T23:21:02.963 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-07T23:21:02.965 INFO:tasks.ceph.osd.3:Started 2023-10-07T23:21:02.965 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-07T23:21:02.965 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-07T23:21:02.970 INFO:tasks.ceph.osd.4:Started 2023-10-07T23:21:02.970 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-07T23:21:02.970 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-07T23:21:02.972 INFO:tasks.ceph.osd.5:Started 2023-10-07T23:21:02.972 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-07T23:21:02.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-07T23:21:02.974 INFO:tasks.ceph.osd.6:Started 2023-10-07T23:21:02.974 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-07T23:21:02.974 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-07T23:21:02.976 INFO:tasks.ceph.osd.7:Started 2023-10-07T23:21:02.976 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:03.023 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.018+0000 7fb40ad3b000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-07T23:21:03.024 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.021+0000 7f5c7800a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:21:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 Processor -- start 2023-10-07T23:21:03.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- start start 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa6cc1532e0 0x7fa6cc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6cc159e30 0x7fa6cc06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6cc0c73f0 con 0x7fa6cc15a280 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc0c67e0 con 0x7fa6cc1532e0 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc0c6ac0 con 0x7fa6cc159e30 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d1389700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa6cc15a280 0x7fa6cc15cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60982/0 (socket says 172.21.15.57:60982) 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa6cc1532e0 0x7fa6cc1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d1389700 1 -- 172.21.15.57:0/3457718670 learned_addr learned my addr 172.21.15.57:0/3457718670 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cbfff700 1 --2- 172.21.15.57:0/3457718670 >> v2:172.21.15.154:3300/0 conn(0x7fa6cc159e30 0x7fa6cc06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:03.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cb7fe700 1 -- 172.21.15.57:0/3457718670 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415456781 0 0) 0x7fa6cc0c73f0 con 0x7fa6cc15a280 2023-10-07T23:21:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cb7fe700 1 -- 172.21.15.57:0/3457718670 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc003570 con 0x7fa6cc15a280 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 -- 172.21.15.57:0/3457718670 >> v2:172.21.15.154:3300/0 conn(0x7fa6cc159e30 msgr2=0x7fa6cc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 --2- 172.21.15.57:0/3457718670 >> v2:172.21.15.154:3300/0 conn(0x7fa6cc159e30 0x7fa6cc06c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 -- 172.21.15.57:0/3457718670 >> v1:172.21.15.57:6789/0 conn(0x7fa6cc15a280 legacy=0x7fa6cc15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 -- 172.21.15.57:0/3457718670 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc0c6da0 con 0x7fa6cc1532e0 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d0b88700 1 --2- 172.21.15.57:0/3457718670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc1536b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0c91d0 tx=0x7fa6c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ccf33895cedcf server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cb7fe700 1 -- 172.21.15.57:0/3457718670 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0011630 con 0x7fa6cc1532e0 2023-10-07T23:21:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cb7fe700 1 -- 172.21.15.57:0/3457718670 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c00117b0 con 0x7fa6cc1532e0 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6cb7fe700 1 -- 172.21.15.57:0/3457718670 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0011b70 con 0x7fa6cc1532e0 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- 172.21.15.57:0/3457718670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc1532e0 msgr2=0x7fa6cc1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/3457718670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc1536b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0c91d0 tx=0x7fa6c00091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- 172.21.15.57:0/3457718670 shutdown_connections 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/3457718670 >> v2:172.21.15.154:3300/0 conn(0x7fa6cc159e30 0x7fa6cc06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/3457718670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc1536b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.086+0000 7fa6d2dec700 1 -- 172.21.15.57:0/3457718670 >> 172.21.15.57:0/3457718670 conn(0x7fa6cc0684c0 msgr2=0x7fa6cc0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.090+0000 7fa6d2dec700 1 -- 172.21.15.57:0/3457718670 shutdown_connections 2023-10-07T23:21:03.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.090+0000 7fa6d2dec700 1 -- 172.21.15.57:0/3457718670 wait complete. 2023-10-07T23:21:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.090+0000 7fa6d2dec700 1 Processor -- start 2023-10-07T23:21:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.090+0000 7fa6d2dec700 1 -- start start 2023-10-07T23:21:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6cc159e30 0x7fa6cc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 0x7fa6cc1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc1518e0 con 0x7fa6cc15a280 2023-10-07T23:21:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0c5890 con 0x7fa6cc1532e0 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0c5ac0 con 0x7fa6cc159e30 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d1389700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 0x7fa6cc1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d1389700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 0x7fa6cc1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42214/0 (socket says 172.21.15.57:42214) 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d1389700 1 -- 172.21.15.57:0/1321661549 learned_addr learned my addr 172.21.15.57:0/1321661549 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6cbfff700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6cc159e30 0x7fa6cc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 -- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6cc159e30 msgr2=0x7fa6cc14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6cc159e30 0x7fa6cc14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 -- 172.21.15.57:0/1321661549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 msgr2=0x7fa6cc1501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 0x7fa6cc1501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 -- 172.21.15.57:0/1321661549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0009420 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d0b88700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc0c5350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0016000 tx=0x7fa6c0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c001f070 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c0002f30 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc151ad0 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0011920 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc152090 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa6c0023070 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.094+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fa6c0011a80 con 0x7fa6cc1532e0 2023-10-07T23:21:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.098+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6cc0711c0 con 0x7fa6cc1532e0 2023-10-07T23:21:03.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.102+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa6c002a070 con 0x7fa6cc1532e0 2023-10-07T23:21:03.151 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.146+0000 7fb40ad3b000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-07T23:21:03.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6cc0c7080 con 0x7fa6cc1532e0 2023-10-07T23:21:03.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6c8ff9700 1 -- 172.21.15.57:0/1321661549 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fa6c0015020 con 0x7fa6cc1532e0 2023-10-07T23:21:03.176 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:03.177 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:02.935726+0000","last_up_change":"0.000000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:03.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 msgr2=0x7fa6cc0c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:03.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc0c5350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0016000 tx=0x7fa6c0017000 comp rx=0 tx=0).stop 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 shutdown_connections 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa6cc15a280 0x7fa6cc1501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6cc159e30 0x7fa6cc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 --2- 172.21.15.57:0/1321661549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6cc1532e0 0x7fa6cc0c5350 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 >> 172.21.15.57:0/1321661549 conn(0x7fa6cc0684c0 msgr2=0x7fa6cc0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.170+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 shutdown_connections 2023-10-07T23:21:03.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:03.174+0000 7fa6d2dec700 1 -- 172.21.15.57:0/1321661549 wait complete. 2023-10-07T23:21:03.189 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-07T23:21:03.189 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-07T23:21:03.331 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-10-07T23:21:03.326+0000 7f5dcbbd13c0 -1 Falling back to public interface 2023-10-07T23:21:03.339 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-10-07T23:21:03.334+0000 7f6c2d80a3c0 -1 Falling back to public interface 2023-10-07T23:21:03.349 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.345+0000 7f5c7800a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:21:03.351 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-10-07T23:21:03.346+0000 7fabad5023c0 -1 Falling back to public interface 2023-10-07T23:21:03.483 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.478+0000 7fb40ad3b000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-07T23:21:03.523 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.520+0000 7f5c7800a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:21:03.561 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-07T23:21:03.556+0000 7f940ab823c0 -1 Falling back to public interface 2023-10-07T23:21:03.577 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-07T23:21:03.572+0000 7f29ba63a3c0 -1 Falling back to public interface 2023-10-07T23:21:03.577 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-07T23:21:03.572+0000 7f0403b523c0 -1 Falling back to public interface 2023-10-07T23:21:03.591 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.584+0000 7f5c7800a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:21:03.604 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-10-07T23:21:03.598+0000 7f621cba53c0 -1 Falling back to public interface 2023-10-07T23:21:03.605 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-07T23:21:03.600+0000 7f8c643a93c0 -1 Falling back to public interface 2023-10-07T23:21:03.643 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.636+0000 7f5c7800a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:21:03.655 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.650+0000 7fb40ad3b000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-07T23:21:03.700 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.696+0000 7f5c7800a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:21:03.718 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.714+0000 7fb40ad3b000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-07T23:21:03.771 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.766+0000 7fb40ad3b000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-07T23:21:03.837 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:03.830+0000 7fb40ad3b000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-07T23:21:03.978 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:03.972+0000 7f5c7800a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:21:03.993 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:04.033 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:04.028+0000 7f5c7800a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:21:04.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 Processor -- start 2023-10-07T23:21:04.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 -- start start 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa7c156950 0x7faa7c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa7c0b2690 0x7faa7c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c159d70 con 0x7faa7c156d20 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0b0c50 con 0x7faa7c0b2690 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.078+0000 7faa829d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c159720 con 0x7faa7c156950 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faa7c0b2690 0x7faa7c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:04.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa821d7700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7faa7c156d20 0x7faa7c06c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60988/0 (socket says 172.21.15.57:60988) 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa821d7700 1 -- 172.21.15.57:0/1052172046 learned_addr learned my addr 172.21.15.57:0/1052172046 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa819d6700 1 --2- 172.21.15.57:0/1052172046 >> v2:172.21.15.154:3300/0 conn(0x7faa7c156950 0x7faa7c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 -- 172.21.15.57:0/1052172046 >> v2:172.21.15.154:3300/0 conn(0x7faa7c156950 msgr2=0x7faa7c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 --2- 172.21.15.57:0/1052172046 >> v2:172.21.15.154:3300/0 conn(0x7faa7c156950 0x7faa7c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 -- 172.21.15.57:0/1052172046 >> v1:172.21.15.57:6789/0 conn(0x7faa7c156d20 legacy=0x7faa7c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa809d4700 1 -- 172.21.15.57:0/1052172046 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387470192 0 0) 0x7faa7c159d70 con 0x7faa7c156d20 2023-10-07T23:21:04.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 -- 172.21.15.57:0/1052172046 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c159a00 con 0x7faa7c0b2690 2023-10-07T23:21:04.088 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa811d5700 1 --2- 172.21.15.57:0/1052172046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c0b0400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0b2b30 tx=0x7faa6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19b27f8bc50bc59 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:04.088 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa809d4700 1 -- 172.21.15.57:0/1052172046 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011630 con 0x7faa7c0b2690 2023-10-07T23:21:04.088 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa809d4700 1 -- 172.21.15.57:0/1052172046 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c011790 con 0x7faa7c0b2690 2023-10-07T23:21:04.088 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa809d4700 1 -- 172.21.15.57:0/1052172046 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011b30 con 0x7faa7c0b2690 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- 172.21.15.57:0/1052172046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 msgr2=0x7faa7c0b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 --2- 172.21.15.57:0/1052172046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c0b0400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0b2b30 tx=0x7faa6c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- 172.21.15.57:0/1052172046 shutdown_connections 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 --2- 172.21.15.57:0/1052172046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c0b0400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 --2- 172.21.15.57:0/1052172046 >> v2:172.21.15.154:3300/0 conn(0x7faa7c156950 0x7faa7c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- 172.21.15.57:0/1052172046 >> 172.21.15.57:0/1052172046 conn(0x7faa7c0ccb10 msgr2=0x7faa7c0ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- 172.21.15.57:0/1052172046 shutdown_connections 2023-10-07T23:21:04.089 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- 172.21.15.57:0/1052172046 wait complete. 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 Processor -- start 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.082+0000 7faa829d8700 1 -- start start 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa7c156950 0x7faa7c0cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa7c156d20 0x7faa7c0cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0a5160 con 0x7faa7c0b2690 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1388c0 con 0x7faa7c156950 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0a4550 con 0x7faa7c156d20 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:04.092 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42230/0 (socket says 172.21.15.57:42230) 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 -- 172.21.15.57:0/523906160 learned_addr learned my addr 172.21.15.57:0/523906160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa811d5700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa7c156950 0x7faa7c0cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa821d7700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa7c156d20 0x7faa7c0cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 -- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa7c156d20 msgr2=0x7faa7c0cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa7c156d20 0x7faa7c0cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 -- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa7c156950 msgr2=0x7faa7c0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa7c156950 0x7faa7c0cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 -- 172.21.15.57:0/523906160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c009420 con 0x7faa7c0b2690 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa819d6700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa74009f20 tx=0x7faa7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400cb70 con 0x7faa7c0b2690 2023-10-07T23:21:04.093 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7400f040 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c0a4830 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74014480 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c0a4df0 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faa740147f0 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7faa74014aa0 con 0x7faa7c0b2690 2023-10-07T23:21:04.094 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.086+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c04be40 con 0x7faa7c0b2690 2023-10-07T23:21:04.096 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.090+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faa74018020 con 0x7faa7c0b2690 2023-10-07T23:21:04.098 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:04.092+0000 7f5c7800a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:21:04.123 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:04.118+0000 7fb40ad3b000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-07T23:21:04.160 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.154+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa7c0b0940 con 0x7faa7c0b2690 2023-10-07T23:21:04.161 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.154+0000 7faa71ffb700 1 -- 172.21.15.57:0/523906160 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7faa74014d20 con 0x7faa7c0b2690 2023-10-07T23:21:04.161 INFO:teuthology.misc.health.smithi057.stdout: 2023-10-07T23:21:04.161 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":10,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:02.935726+0000","last_up_change":"0.000000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 msgr2=0x7faa7c138380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faa74009f20 tx=0x7faa7400b040 comp rx=0 tx=0).stop 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 shutdown_connections 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faa7c156d20 0x7faa7c0cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa7c156950 0x7faa7c0cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 --2- 172.21.15.57:0/523906160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa7c0b2690 0x7faa7c138380 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 >> 172.21.15.57:0/523906160 conn(0x7faa7c0ccb10 msgr2=0x7faa7c0c6330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 shutdown_connections 2023-10-07T23:21:04.163 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:04.158+0000 7faa829d8700 1 -- 172.21.15.57:0/523906160 wait complete. 2023-10-07T23:21:04.172 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-07T23:21:04.183 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:04.178+0000 7fb40ad3b000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-07T23:21:04.252 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:04.246+0000 7fb40ad3b000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-07T23:21:04.257 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:04.252+0000 7f5c7800a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:21:04.324 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:04.320+0000 7f5c7800a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:21:04.423 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:04.418+0000 7fb40ad3b000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-07T23:21:04.497 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:04.490+0000 7fb40ad3b000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-07T23:21:05.029 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:05.024+0000 7f5c7800a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:21:05.242 INFO:tasks.ceph.mgr.y.smithi057.stderr:2023-10-07T23:21:05.238+0000 7fb40ad3b000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-07T23:21:06.079 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:06.072+0000 7f5c61102700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-07T23:21:06.079 INFO:tasks.ceph.mgr.x.smithi154.stderr:2023-10-07T23:21:06.072+0000 7f5c61102700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-07T23:21:07.006 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-10-07T23:21:06.998+0000 7fabad5023c0 -1 osd.1 0 log_to_monitors true 2023-10-07T23:21:07.188 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-07T23:21:07.184+0000 7f29ba63a3c0 -1 osd.4 0 log_to_monitors true 2023-10-07T23:21:07.251 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-10-07T23:21:07.246+0000 7f5dcbbd13c0 -1 osd.2 0 log_to_monitors true 2023-10-07T23:21:07.268 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-10-07T23:21:07.262+0000 7f6c2d80a3c0 -1 osd.3 0 log_to_monitors true 2023-10-07T23:21:07.379 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-07T23:21:07.372+0000 7f0403b523c0 -1 osd.7 0 log_to_monitors true 2023-10-07T23:21:07.408 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-07T23:21:07.404+0000 7f940ab823c0 -1 osd.6 0 log_to_monitors true 2023-10-07T23:21:07.473 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-10-07T23:21:07.470+0000 7f621cba53c0 -1 osd.0 0 log_to_monitors true 2023-10-07T23:21:07.485 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-07T23:21:07.480+0000 7f8c643a93c0 -1 osd.5 0 log_to_monitors true 2023-10-07T23:21:09.582 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-10-07T23:21:09.578+0000 7faba8ca2700 -1 osd.1 0 waiting for initial osdmap 2023-10-07T23:21:09.625 INFO:tasks.ceph.osd.1.smithi057.stderr:2023-10-07T23:21:09.622+0000 7faba1a46700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:10.797 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-10-07T23:21:10.790+0000 7f5dc7371700 -1 osd.2 0 waiting for initial osdmap 2023-10-07T23:21:10.798 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-10-07T23:21:10.794+0000 7f6218345700 -1 osd.0 0 waiting for initial osdmap 2023-10-07T23:21:10.802 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-10-07T23:21:10.798+0000 7f6c2a7cf700 -1 osd.3 0 waiting for initial osdmap 2023-10-07T23:21:10.803 INFO:tasks.ceph.osd.2.smithi057.stderr:2023-10-07T23:21:10.798+0000 7f5dc0115700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:10.804 INFO:tasks.ceph.osd.0.smithi057.stderr:2023-10-07T23:21:10.798+0000 7f62110e9700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:10.811 INFO:tasks.ceph.osd.3.smithi057.stderr:2023-10-07T23:21:10.806+0000 7f6c21d4e700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:10.978 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:11.076 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 Processor -- start 2023-10-07T23:21:11.076 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 -- start start 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fec440c7e70 0x7fec440ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440c1220 con 0x7fec440c7aa0 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440cb720 con 0x7fec44159200 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4ad43700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440c6fa0 con 0x7fec440c7e70 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec49d41700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fec440c7aa0 0x7fec44158af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45426/0 (socket says 172.21.15.57:45426) 2023-10-07T23:21:11.078 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec49d41700 1 -- 172.21.15.57:0/768321413 learned_addr learned my addr 172.21.15.57:0/768321413 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4a542700 1 --2- 172.21.15.57:0/768321413 >> v2:172.21.15.57:3300/0 conn(0x7fec440c7e70 0x7fec440ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972379860 0 0) 0x7fec440c1220 con 0x7fec440c7aa0 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec30003570 con 0x7fec440c7aa0 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204173636 0 0) 0x7fec440cb720 con 0x7fec44159200 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec440c1220 con 0x7fec44159200 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4a542700 1 -- 172.21.15.57:0/768321413 >> v1:172.21.15.154:6789/0 conn(0x7fec44159200 legacy=0x7fec440c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.070+0000 7fec4a542700 1 -- 172.21.15.57:0/768321413 >> v1:172.21.15.57:6789/0 conn(0x7fec440c7aa0 legacy=0x7fec44158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4a542700 1 -- 172.21.15.57:0/768321413 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440c08f0 con 0x7fec440c7e70 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4a542700 1 --2- 172.21.15.57:0/768321413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec440c7e70 0x7fec440ae8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e60 tx=0x7fec3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6227b42130a8a177 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00e070 con 0x7fec440c7e70 2023-10-07T23:21:11.079 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00bab0 con 0x7fec440c7e70 2023-10-07T23:21:11.080 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec48d3f700 1 -- 172.21.15.57:0/768321413 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00be70 con 0x7fec440c7e70 2023-10-07T23:21:11.080 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- 172.21.15.57:0/768321413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec440c7e70 msgr2=0x7fec440ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 --2- 172.21.15.57:0/768321413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec440c7e70 0x7fec440ae8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e60 tx=0x7fec3c009780 comp rx=0 tx=0).stop 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- 172.21.15.57:0/768321413 shutdown_connections 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 --2- 172.21.15.57:0/768321413 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec440c7e70 0x7fec440ae8d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- 172.21.15.57:0/768321413 >> 172.21.15.57:0/768321413 conn(0x7fec440c8430 msgr2=0x7fec440c8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- 172.21.15.57:0/768321413 shutdown_connections 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- 172.21.15.57:0/768321413 wait complete. 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 Processor -- start 2023-10-07T23:21:11.081 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 -- start start 2023-10-07T23:21:11.083 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.074+0000 7fec4ad43700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:11.083 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec440c7e70 0x7fec440aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b0620 con 0x7fec44159200 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b1610 con 0x7fec440c7aa0 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440afa10 con 0x7fec440c7e70 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec49d41700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec49d41700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:40914/0 (socket says 172.21.15.57:40914) 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec49d41700 1 -- 172.21.15.57:0/516971646 learned_addr learned my addr 172.21.15.57:0/516971646 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40000/0 (socket says 172.21.15.57:40000) 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec49540700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec440c7e70 0x7fec440aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:11.084 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 -- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec440c7e70 msgr2=0x7fec440aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec440c7e70 0x7fec440aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 -- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 msgr2=0x7fec4413ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 -- 172.21.15.57:0/516971646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009a20 con 0x7fec44159200 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec49d41700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4a542700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec440c9540 tx=0x7fec3c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c012db0 con 0x7fec44159200 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c026480 con 0x7fec44159200 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec440afcf0 con 0x7fec44159200 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c026840 con 0x7fec44159200 2023-10-07T23:21:11.085 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec440b02b0 con 0x7fec44159200 2023-10-07T23:21:11.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.078+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec440b07d0 con 0x7fec44159200 2023-10-07T23:21:11.086 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.082+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) v1 ==== 79903+0+0 (secure 0 0 0) 0x7fec3c01c400 con 0x7fec44159200 2023-10-07T23:21:11.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.082+0000 7fec39ffb700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 0x7fec28062970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:11.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.082+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2507+0+0 (secure 0 0 0) 0x7fec3c07d8e0 con 0x7fec44159200 2023-10-07T23:21:11.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.082+0000 7fec49d41700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 0x7fec28062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:11.087 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.082+0000 7fec49d41700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 0x7fec28062970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec400058d0 tx=0x7fec40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:11.095 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.090+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fec440b07d0 con 0x7fec44159200 2023-10-07T23:21:11.255 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.250+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec440b07d0 con 0x7fec44159200 2023-10-07T23:21:11.256 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.250+0000 7fec39ffb700 1 -- 172.21.15.57:0/516971646 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7450 (secure 0 0 0) 0x7fec440b07d0 con 0x7fec44159200 2023-10-07T23:21:11.257 INFO:teuthology.misc.health.smithi057.stdout: 2023-10-07T23:21:11.257 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":14,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:10.116908+0000","last_up_change":"2023-10-07T23:21:10.116908+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 msgr2=0x7fec28062970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 0x7fec28062970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec400058d0 tx=0x7fec40009040 comp rx=0 tx=0).stop 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 msgr2=0x7fec440af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec440c9540 tx=0x7fec3c019000 comp rx=0 tx=0).stop 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 shutdown_connections 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec44159200 0x7fec440af3e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.259 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec440c7e70 0x7fec440aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.260 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec280604c0 0x7fec28062970 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.260 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 --2- 172.21.15.57:0/516971646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec440c7aa0 0x7fec4413ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:11.260 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 >> 172.21.15.57:0/516971646 conn(0x7fec440c8430 msgr2=0x7fec44069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:11.260 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 shutdown_connections 2023-10-07T23:21:11.260 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:11.254+0000 7fec4ad43700 1 -- 172.21.15.57:0/516971646 wait complete. 2023-10-07T23:21:11.269 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-10-07T23:21:11.400 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-07T23:21:11.396+0000 7f03ff2f2700 -1 osd.7 0 waiting for initial osdmap 2023-10-07T23:21:11.407 INFO:tasks.ceph.osd.7.smithi154.stderr:2023-10-07T23:21:11.400+0000 7f03f8096700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:11.421 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-07T23:21:11.416+0000 7f8c5fb49700 -1 osd.5 0 waiting for initial osdmap 2023-10-07T23:21:11.428 INFO:tasks.ceph.osd.5.smithi154.stderr:2023-10-07T23:21:11.424+0000 7f8c588ed700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:11.430 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-07T23:21:11.424+0000 7f9406322700 -1 osd.6 0 waiting for initial osdmap 2023-10-07T23:21:11.431 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-07T23:21:11.424+0000 7f29b5dda700 -1 osd.4 0 waiting for initial osdmap 2023-10-07T23:21:11.436 INFO:tasks.ceph.osd.6.smithi154.stderr:2023-10-07T23:21:11.432+0000 7f93ff0c6700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:11.437 INFO:tasks.ceph.osd.4.smithi154.stderr:2023-10-07T23:21:11.432+0000 7f29aeb7e700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-07T23:21:18.074 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:18.182 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 Processor -- start 2023-10-07T23:21:18.183 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 -- start start 2023-10-07T23:21:18.185 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3e0c0b1780 0x7f3e0c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.185 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3e0c0af060 0x7f3e0c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.185 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0c8d30 0x7f3e0c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.185 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c0ca8f0 con 0x7f3e0c0af060 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c1516e0 con 0x7f3e0c0b1780 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e1322b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0caa80 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e11a28700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3e0c0af060 0x7f3e0c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e11a28700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3e0c0af060 0x7f3e0c0b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42746/0 (socket says 172.21.15.57:42746) 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e11a28700 1 -- 172.21.15.57:0/2268158026 learned_addr learned my addr 172.21.15.57:0/2268158026 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e12a2a700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0c8d30 0x7f3e0c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.186 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.178+0000 7f3e12229700 1 --2- 172.21.15.57:0/2268158026 >> v2:172.21.15.154:3300/0 conn(0x7f3e0c0b1780 0x7f3e0c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.187 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 -- 172.21.15.57:0/2268158026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0c8d30 msgr2=0x7f3e0c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:18.187 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0c8d30 0x7f3e0c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 -- 172.21.15.57:0/2268158026 >> v2:172.21.15.154:3300/0 conn(0x7f3e0c0b1780 msgr2=0x7f3e0c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 --2- 172.21.15.57:0/2268158026 >> v2:172.21.15.154:3300/0 conn(0x7f3e0c0b1780 0x7f3e0c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 -- 172.21.15.57:0/2268158026 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0b2090 con 0x7f3e0c0af060 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11a28700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0af060 0x7f3e0c0b1b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c06b560 tx=0x7f3dfc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c427fbff2a3f58 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11227700 1 -- 172.21.15.57:0/2268158026 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00b640 con 0x7f3e0c0af060 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11227700 1 -- 172.21.15.57:0/2268158026 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc00b7a0 con 0x7f3e0c0af060 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e11227700 1 -- 172.21.15.57:0/2268158026 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00bb60 con 0x7f3e0c0af060 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- 172.21.15.57:0/2268158026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0af060 msgr2=0x7f3e0c0b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0af060 0x7f3e0c0b1b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c06b560 tx=0x7f3dfc009250 comp rx=0 tx=0).stop 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- 172.21.15.57:0/2268158026 shutdown_connections 2023-10-07T23:21:18.188 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0c8d30 0x7f3e0c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.189 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2268158026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0af060 0x7f3e0c0b1b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.189 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2268158026 >> v2:172.21.15.154:3300/0 conn(0x7f3e0c0b1780 0x7f3e0c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.189 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- 172.21.15.57:0/2268158026 >> 172.21.15.57:0/2268158026 conn(0x7f3e0c0cc2a0 msgr2=0x7f3e0c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.189 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- 172.21.15.57:0/2268158026 shutdown_connections 2023-10-07T23:21:18.189 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- 172.21.15.57:0/2268158026 wait complete. 2023-10-07T23:21:18.190 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 Processor -- start 2023-10-07T23:21:18.190 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.182+0000 7f3e1322b700 1 -- start start 2023-10-07T23:21:18.191 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 0x7f3e0c1418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0b1780 0x7f3e0c068300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c067e70 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c067260 con 0x7f3e0c0af060 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c067540 con 0x7f3e0c0b1780 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42750/0 (socket says 172.21.15.57:42750) 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 -- 172.21.15.57:0/2292476947 learned_addr learned my addr 172.21.15.57:0/2292476947 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12229700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 0x7f3e0c1418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e11a28700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0b1780 0x7f3e0c068300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.192 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 -- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0b1780 msgr2=0x7f3e0c068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0b1780 0x7f3e0c068300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 -- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 msgr2=0x7f3e0c1418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 0x7f3e0c1418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 -- 172.21.15.57:0/2292476947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0094f0 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12229700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 0x7f3e0c1418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e12a2a700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009df0 tx=0x7f3e0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400cb50 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0400f040 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c067820 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04014480 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.193 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c141de0 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.194 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.186+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c068020 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.194 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.190+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3e040147f0 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.194 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.190+0000 7f3e027fc700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 0x7f3df0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.194 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.190+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3e0407d120 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.195 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.190+0000 7f3e12229700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 0x7f3df0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.195 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.190+0000 7f3e12229700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 0x7f3df0062a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae950 tx=0x7f3e08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.202 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.198+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3e0404da60 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.367 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.362+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e0c04b600 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.369 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.362+0000 7f3e027fc700 1 -- 172.21.15.57:0/2292476947 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f3e04051080 con 0x7f3e0c0c8d30 2023-10-07T23:21:18.369 INFO:teuthology.misc.health.smithi057.stdout: 2023-10-07T23:21:18.369 INFO:teuthology.misc.health.smithi057.stdout:{"epoch":19,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:15.837204+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 msgr2=0x7f3df0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 0x7f3df0062a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ae950 tx=0x7f3e08008040 comp rx=0 tx=0).stop 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 msgr2=0x7f3e0c16a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009df0 tx=0x7f3e0400b040 comp rx=0 tx=0).stop 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 shutdown_connections 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e0c0c8d30 0x7f3e0c16a170 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3e0c0b1780 0x7f3e0c068300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.372 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3df0060580 0x7f3df0062a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.373 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 --2- 172.21.15.57:0/2292476947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e0c0af060 0x7f3e0c1418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.373 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 >> 172.21.15.57:0/2292476947 conn(0x7f3e0c0cc2a0 msgr2=0x7f3e0c0af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.373 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 shutdown_connections 2023-10-07T23:21:18.373 INFO:teuthology.misc.health.smithi057.stderr:2023-10-07T23:21:18.366+0000 7f3e1322b700 1 -- 172.21.15.57:0/2292476947 wait complete. 2023-10-07T23:21:18.383 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-07T23:21:18.383 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-07T23:21:18.383 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T23:21:18.383 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T23:21:18.386 INFO:tasks.ceph.mds.a:Started 2023-10-07T23:21:18.386 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T23:21:18.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T23:21:18.388 INFO:tasks.ceph.mds.c:Started 2023-10-07T23:21:18.388 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T23:21:18.388 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T23:21:18.390 INFO:tasks.ceph.mds.b:Started 2023-10-07T23:21:18.390 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T23:21:18.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T23:21:18.393 INFO:tasks.ceph.mds.d:Started 2023-10-07T23:21:18.393 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-07T23:21:18.393 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:18.461 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-10-07T23:21:18.475 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-10-07T23:21:18.477 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-07T23:21:18.481 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-07T23:21:18.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 Processor -- start 2023-10-07T23:21:18.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 -- start start 2023-10-07T23:21:18.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 0x7ff5d406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5d40c93c0 0x7ff5d40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d41594f0 con 0x7ff5d414fd80 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40c9da0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4159790 con 0x7ff5d4150150 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d2ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff5d414fd80 0x7ff5d40abb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54574/0 (socket says 172.21.15.57:54574) 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d2ffd700 1 -- 172.21.15.57:0/523089611 learned_addr learned my addr 172.21.15.57:0/523089611 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/523089611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 0x7ff5d406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d37fe700 1 --2- 172.21.15.57:0/523089611 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40c93c0 0x7ff5d40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 -- 172.21.15.57:0/523089611 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40c93c0 msgr2=0x7ff5d40c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:18.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/523089611 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40c93c0 0x7ff5d40c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 -- 172.21.15.57:0/523089611 >> v1:172.21.15.57:6789/0 conn(0x7ff5d414fd80 legacy=0x7ff5d40abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 -- 172.21.15.57:0/523089611 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4159a70 con 0x7ff5d4150150 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/523089611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 0x7ff5d406c800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098e0 tx=0x7ff5c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2d5a07e630da10 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d1ffb700 1 -- 172.21.15.57:0/523089611 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800b640 con 0x7ff5d4150150 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d1ffb700 1 -- 172.21.15.57:0/523089611 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800b7a0 con 0x7ff5d4150150 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d1ffb700 1 -- 172.21.15.57:0/523089611 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800bb40 con 0x7ff5d4150150 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 -- 172.21.15.57:0/523089611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 msgr2=0x7ff5d406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.486+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/523089611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 0x7ff5d406c800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098e0 tx=0x7ff5c80091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- 172.21.15.57:0/523089611 shutdown_connections 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/523089611 >> v2:172.21.15.154:3300/0 conn(0x7ff5d40c93c0 0x7ff5d40c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/523089611 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d4150150 0x7ff5d406c800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- 172.21.15.57:0/523089611 >> 172.21.15.57:0/523089611 conn(0x7ff5d4067b80 msgr2=0x7ff5d40c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- 172.21.15.57:0/523089611 shutdown_connections 2023-10-07T23:21:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- 172.21.15.57:0/523089611 wait complete. 2023-10-07T23:21:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 Processor -- start 2023-10-07T23:21:18.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- start start 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d414fd80 0x7ff5d40af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 0x7ff5d413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d41514c0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d41505d0 con 0x7ff5d4150150 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d41508b0 con 0x7ff5d414fd80 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42778/0 (socket says 172.21.15.57:42778) 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 -- 172.21.15.57:0/1171063543 learned_addr learned my addr 172.21.15.57:0/1171063543 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:18.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d37fe700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 0x7ff5d413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d414fd80 0x7ff5d40af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 -- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d414fd80 msgr2=0x7ff5d40af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d414fd80 0x7ff5d40af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 -- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 msgr2=0x7ff5d413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 0x7ff5d413fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d2ffd700 1 -- 172.21.15.57:0/1171063543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009420 con 0x7ff5d40c93c0 2023-10-07T23:21:18.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.490+0000 7ff5d37fe700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 0x7ff5d413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:18.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5d2ffd700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cd30 tx=0x7ff5cc001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00ce30 con 0x7ff5d40c93c0 2023-10-07T23:21:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d4150b90 con 0x7ff5d40c93c0 2023-10-07T23:21:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4151150 con 0x7ff5d40c93c0 2023-10-07T23:21:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc004dc0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.494+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00aac0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.498+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d40ad5e0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.498+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff5cc0070f0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.502+0000 7ff5baffd700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 0x7ff5c0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.502+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff5cc07c790 con 0x7ff5d40c93c0 2023-10-07T23:21:18.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.502+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 0x7ff5c0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.502+0000 7ff5d27fc700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 0x7ff5c0062a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098e0 tx=0x7ff5c8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.510+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff5cc04d200 con 0x7ff5d40c93c0 2023-10-07T23:21:18.641 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:21:18.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:21:18.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.674+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5d40ad5e0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.674+0000 7ff5baffd700 1 -- 172.21.15.57:0/1171063543 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12127 (secure 0 0 0) 0x7ff5d40ad5e0 con 0x7ff5d40c93c0 2023-10-07T23:21:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:18.680 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:15.837204+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:18.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 msgr2=0x7ff5c0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 0x7ff5c0062a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80098e0 tx=0x7ff5c8017000 comp rx=0 tx=0).stop 2023-10-07T23:21:18.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 msgr2=0x7ff5d41404d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00cd30 tx=0x7ff5cc001ad0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 shutdown_connections 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff5c0060550 0x7ff5c0062a00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5d4150150 0x7ff5d413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5d414fd80 0x7ff5d40af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 --2- 172.21.15.57:0/1171063543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5d40c93c0 0x7ff5d41404d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 >> 172.21.15.57:0/1171063543 conn(0x7ff5d4067b80 msgr2=0x7ff5d414e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 shutdown_connections 2023-10-07T23:21:18.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.678+0000 7ff5d3fff700 1 -- 172.21.15.57:0/1171063543 wait complete. 2023-10-07T23:21:18.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:21:18.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.806+0000 7fc85e027700 1 Processor -- start 2023-10-07T23:21:18.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.806+0000 7fc85e027700 1 -- start start 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc85e027700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc85806dbd0 0x7fc8580c7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc85e027700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 0x7fc85806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc85e027700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc858068300 con 0x7fc8580ae3d0 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc85e027700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85806d1c0 con 0x7fc85806dbd0 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc85e027700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580688c0 con 0x7fc85806c800 2023-10-07T23:21:18.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc857fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 0x7fc85806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc85806dbd0 0x7fc8580c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc857fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 0x7fc85806cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:51354/0 (socket says 172.21.15.57:51354) 2023-10-07T23:21:18.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc85806dbd0 0x7fc8580c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:56824/0 (socket says 172.21.15.57:56824) 2023-10-07T23:21:18.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc857fff700 1 -- 172.21.15.57:0/4199255984 learned_addr learned my addr 172.21.15.57:0/4199255984 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 -- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 msgr2=0x7fc85806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 --2- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 0x7fc85806cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 -- 172.21.15.57:0/4199255984 >> v1:172.21.15.154:6789/0 conn(0x7fc8580ae3d0 legacy=0x7fc8580ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 -- 172.21.15.57:0/4199255984 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580aca50 con 0x7fc85806dbd0 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc84effd700 1 --2- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806dbd0 0x7fc8580c7390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098e0 tx=0x7fc8400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26db3a616a3f6fd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc856ffd700 1 -- 172.21.15.57:0/4199255984 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000b640 con 0x7fc85806dbd0 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc856ffd700 1 -- 172.21.15.57:0/4199255984 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84000b7a0 con 0x7fc85806dbd0 2023-10-07T23:21:18.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.810+0000 7fc856ffd700 1 -- 172.21.15.57:0/4199255984 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000bb40 con 0x7fc85806dbd0 2023-10-07T23:21:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806dbd0 msgr2=0x7fc8580c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 --2- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806dbd0 0x7fc8580c7390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098e0 tx=0x7fc8400091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- 172.21.15.57:0/4199255984 shutdown_connections 2023-10-07T23:21:18.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 --2- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc85806c800 0x7fc85806cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 --2- 172.21.15.57:0/4199255984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806dbd0 0x7fc8580c7390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- 172.21.15.57:0/4199255984 >> 172.21.15.57:0/4199255984 conn(0x7fc8580ccf10 msgr2=0x7fc8580c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- 172.21.15.57:0/4199255984 shutdown_connections 2023-10-07T23:21:18.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- 172.21.15.57:0/4199255984 wait complete. 2023-10-07T23:21:18.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 Processor -- start 2023-10-07T23:21:18.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.814+0000 7fc85e027700 1 -- start start 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806c800 0x7fc8581596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8580ae3d0 0x7fc858169350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858159fd0 con 0x7fc85806dbd0 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85815a2b0 con 0x7fc85806c800 2023-10-07T23:21:18.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85815a590 con 0x7fc8580ae3d0 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42788/0 (socket says 172.21.15.57:42788) 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 -- 172.21.15.57:0/3227507571 learned_addr learned my addr 172.21.15.57:0/3227507571 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc8577fe700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806c800 0x7fc8581596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc857fff700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8580ae3d0 0x7fc858169350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 -- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8580ae3d0 msgr2=0x7fc858169350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8580ae3d0 0x7fc858169350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 -- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806c800 msgr2=0x7fc8581596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806c800 0x7fc8581596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 -- 172.21.15.57:0/3227507571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840009420 con 0x7fc85806dbd0 2023-10-07T23:21:18.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84effd700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8400093f0 tx=0x7fc840017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84001b960 con 0x7fc85806dbd0 2023-10-07T23:21:18.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84001bae0 con 0x7fc85806dbd0 2023-10-07T23:21:18.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84001bea0 con 0x7fc85806dbd0 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85815a870 con 0x7fc85806dbd0 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.818+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858159c10 con 0x7fc85806dbd0 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8580647b0 con 0x7fc85806dbd0 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc840023070 con 0x7fc85806dbd0 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc84ffff700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 0x7fc838062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:18.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc84007dfd0 con 0x7fc85806dbd0 2023-10-07T23:21:18.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc8577fe700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 0x7fc838062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:18.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.822+0000 7fc8577fe700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 0x7fc838062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc848003a80 tx=0x7fc848007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:18.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.830+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc8400523a0 con 0x7fc85806dbd0 2023-10-07T23:21:18.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8580c79d0 con 0x7fc85806dbd0 2023-10-07T23:21:18.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc84ffff700 1 -- 172.21.15.57:0/3227507571 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8400330a0 con 0x7fc85806dbd0 2023-10-07T23:21:18.995 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:18.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 msgr2=0x7fc838062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 0x7fc838062a00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc848003a80 tx=0x7fc848007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 msgr2=0x7fc858156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8400093f0 tx=0x7fc840017000 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 shutdown_connections 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc8580ae3d0 0x7fc858169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc85806dbd0 0x7fc858156ae0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc838060550 0x7fc838062a00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 --2- 172.21.15.57:0/3227507571 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc85806c800 0x7fc8581596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 >> 172.21.15.57:0/3227507571 conn(0x7fc8580ccf10 msgr2=0x7fc8580b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 shutdown_connections 2023-10-07T23:21:18.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:18.990+0000 7fc85e027700 1 -- 172.21.15.57:0/3227507571 wait complete. 2023-10-07T23:21:19.009 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:19.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.110+0000 7f3bd0cb0700 1 Processor -- start 2023-10-07T23:21:19.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.110+0000 7f3bd0cb0700 1 -- start start 2023-10-07T23:21:19.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3bcc15c060 0x7f3bcc0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 0x7f3bcc0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0ca4b0 con 0x7f3bcc0cc960 2023-10-07T23:21:19.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cda10 con 0x7f3bcc15c060 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cce00 con 0x7f3bcc15c430 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcaffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3bcc0cc960 0x7f3bcc0c9da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54586/0 (socket says 172.21.15.57:54586) 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcaffd700 1 -- 172.21.15.57:0/3807979331 learned_addr learned my addr 172.21.15.57:0/3807979331 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/3807979331 >> v2:172.21.15.57:3300/0 conn(0x7f3bcc15c060 0x7f3bcc0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 0x7f3bcc0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3807979331 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056029247 0 0) 0x7f3bcc0ca4b0 con 0x7f3bcc0cc960 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3807979331 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb4003570 con 0x7f3bcc0cc960 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3807979331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 msgr2=0x7f3bcc0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 0x7f3bcc0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3807979331 >> v1:172.21.15.57:6789/0 conn(0x7f3bcc0cc960 legacy=0x7f3bcc0c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 -- 172.21.15.57:0/3807979331 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0cd0e0 con 0x7f3bcc15c060 2023-10-07T23:21:19.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 0x7f3bcc0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:19.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c060 0x7f3bcc0c7510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098d0 tx=0x7f3bbc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d066d0e0dab05a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:19.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3807979331 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc005c10 con 0x7f3bcc15c060 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3807979331 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc005d70 con 0x7f3bcc15c060 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bca7fc700 1 -- 172.21.15.57:0/3807979331 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc006280 con 0x7f3bcc15c060 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/3807979331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c060 msgr2=0x7f3bcc0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c060 0x7f3bcc0c7510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0098d0 tx=0x7f3bbc009120 comp rx=0 tx=0).stop 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/3807979331 shutdown_connections 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c430 0x7f3bcc0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/3807979331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c060 0x7f3bcc0c7510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/3807979331 >> 172.21.15.57:0/3807979331 conn(0x7f3bcc06b5d0 msgr2=0x7f3bcc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:19.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/3807979331 shutdown_connections 2023-10-07T23:21:19.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.114+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/3807979331 wait complete. 2023-10-07T23:21:19.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 Processor -- start 2023-10-07T23:21:19.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 -- start start 2023-10-07T23:21:19.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bcc0cc960 0x7f3bcc159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c060 0x7f3bcc0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0af2f0 con 0x7f3bcc15c430 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0b1f10 con 0x7f3bcc0cc960 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bd0cb0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0adcb0 con 0x7f3bcc15c060 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42806/0 (socket says 172.21.15.57:42806) 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 -- 172.21.15.57:0/2509208746 learned_addr learned my addr 172.21.15.57:0/2509208746 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcaffd700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c060 0x7f3bcc0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bcc0cc960 0x7f3bcc159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 -- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c060 msgr2=0x7f3bcc0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c060 0x7f3bcc0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 -- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bcc0cc960 msgr2=0x7f3bcc159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bcc0cc960 0x7f3bcc159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 -- 172.21.15.57:0/2509208746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0093c0 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.118+0000 7f3bcbfff700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009fa0 tx=0x7f3bc40099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc400dbb0 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc400f040 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0b2790 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4015400 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc0ad660 con 0x7f3bcc15c430 2023-10-07T23:21:19.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc0ade60 con 0x7f3bcc15c430 2023-10-07T23:21:19.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3bc40157f0 con 0x7f3bcc15c430 2023-10-07T23:21:19.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 0x7f3bb8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:19.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3bc407cb90 con 0x7f3bcc15c430 2023-10-07T23:21:19.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.122+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 0x7f3bb8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:19.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.126+0000 7f3bcb7fe700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 0x7f3bb8062a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009160 tx=0x7f3bbc016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:19.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.130+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3bcc0ade60 con 0x7f3bcc15c430 2023-10-07T23:21:19.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.330+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bcc0ade20 con 0x7f3bcc15c430 2023-10-07T23:21:19.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.330+0000 7f3bb37fe700 1 -- 172.21.15.57:0/2509208746 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f3bc404d4d0 con 0x7f3bcc15c430 2023-10-07T23:21:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:19.336 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 msgr2=0x7f3bb8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 0x7f3bb8062a00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009160 tx=0x7f3bbc016000 comp rx=0 tx=0).stop 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 msgr2=0x7f3bcc0b2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009fa0 tx=0x7f3bc40099b0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 shutdown_connections 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3bcc15c430 0x7f3bcc0b2250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3bcc15c060 0x7f3bcc0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3bb8060550 0x7f3bb8062a00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 --2- 172.21.15.57:0/2509208746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bcc0cc960 0x7f3bcc159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:19.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 >> 172.21.15.57:0/2509208746 conn(0x7f3bcc06b5d0 msgr2=0x7f3bcc159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:19.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 shutdown_connections 2023-10-07T23:21:19.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:19.334+0000 7f3bd0cb0700 1 -- 172.21.15.57:0/2509208746 wait complete. 2023-10-07T23:21:19.340 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 2 2023-10-07T23:21:21.351 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:21.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 Processor -- start 2023-10-07T23:21:21.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 -- start start 2023-10-07T23:21:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2d200cba50 0x7f2d200cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2d200c6c10 0x7f2d200c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200cac40 con 0x7f2d200bee60 2023-10-07T23:21:21.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200cc3c0 con 0x7f2d200c6c10 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d24f13700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200ca030 con 0x7f2d200cba50 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2d200c6c10 0x7f2d200c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2d200c6c10 0x7f2d200c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42808/0 (socket says 172.21.15.57:42808) 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/756491830 learned_addr learned my addr 172.21.15.57:0/756491830 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1dd9b700 1 --2- 172.21.15.57:0/756491830 >> v2:172.21.15.154:3300/0 conn(0x7f2d200cba50 0x7f2d200cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/756491830 >> v2:172.21.15.154:3300/0 conn(0x7f2d200cba50 msgr2=0x7f2d200cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 --2- 172.21.15.57:0/756491830 >> v2:172.21.15.154:3300/0 conn(0x7f2d200cba50 0x7f2d200cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/756491830 >> v1:172.21.15.154:6789/0 conn(0x7f2d200bee60 legacy=0x7f2d200cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/756491830 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200ca310 con 0x7f2d200c6c10 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1ed9d700 1 --2- 172.21.15.57:0/756491830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200c6c10 0x7f2d200c7000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b1c20 tx=0x7f2d10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee909c88f9b437 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1d59a700 1 -- 172.21.15.57:0/756491830 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10009e70 con 0x7f2d200c6c10 2023-10-07T23:21:21.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1d59a700 1 -- 172.21.15.57:0/756491830 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000b860 con 0x7f2d200c6c10 2023-10-07T23:21:21.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.462+0000 7f2d1d59a700 1 -- 172.21.15.57:0/756491830 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000bc00 con 0x7f2d200c6c10 2023-10-07T23:21:21.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- 172.21.15.57:0/756491830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200c6c10 msgr2=0x7f2d200c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 --2- 172.21.15.57:0/756491830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200c6c10 0x7f2d200c7000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d200b1c20 tx=0x7f2d10009250 comp rx=0 tx=0).stop 2023-10-07T23:21:21.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- 172.21.15.57:0/756491830 shutdown_connections 2023-10-07T23:21:21.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 --2- 172.21.15.57:0/756491830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200c6c10 0x7f2d200c7000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 --2- 172.21.15.57:0/756491830 >> v2:172.21.15.154:3300/0 conn(0x7f2d200cba50 0x7f2d200cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- 172.21.15.57:0/756491830 >> 172.21.15.57:0/756491830 conn(0x7f2d200670e0 msgr2=0x7f2d200674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:21.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- 172.21.15.57:0/756491830 shutdown_connections 2023-10-07T23:21:21.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- 172.21.15.57:0/756491830 wait complete. 2023-10-07T23:21:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 Processor -- start 2023-10-07T23:21:21.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.466+0000 7f2d24f13700 1 -- start start 2023-10-07T23:21:21.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200bee60 0x7f2d200af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200c6c10 0x7f2d20171140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200bf600 con 0x7f2d200cba50 2023-10-07T23:21:21.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200bf8a0 con 0x7f2d200bee60 2023-10-07T23:21:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200b00e0 con 0x7f2d200c6c10 2023-10-07T23:21:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1e59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200bee60 0x7f2d200af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42818/0 (socket says 172.21.15.57:42818) 2023-10-07T23:21:21.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/1360219932 learned_addr learned my addr 172.21.15.57:0/1360219932 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1dd9b700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200c6c10 0x7f2d20171140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200c6c10 msgr2=0x7f2d20171140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200c6c10 0x7f2d20171140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200bee60 msgr2=0x7f2d200af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200bee60 0x7f2d200af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 -- 172.21.15.57:0/1360219932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100094f0 con 0x7f2d200cba50 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d1ed9d700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d20067e20 tx=0x7f2d10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000b9d0 con 0x7f2d200cba50 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000bb30 con 0x7f2d200cba50 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200b03c0 con 0x7f2d200cba50 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10025670 con 0x7f2d200cba50 2023-10-07T23:21:21.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.470+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2015ceb0 con 0x7f2d200cba50 2023-10-07T23:21:21.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d200711c0 con 0x7f2d200cba50 2023-10-07T23:21:21.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2d10023070 con 0x7f2d200cba50 2023-10-07T23:21:21.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d0e7fc700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 0x7f2d00062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2d1007da60 con 0x7f2d200cba50 2023-10-07T23:21:21.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d1e59c700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 0x7f2d00062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.474+0000 7f2d1e59c700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 0x7f2d00062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003aa0 tx=0x7f2d14000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.482+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2d1004e3a0 con 0x7f2d200cba50 2023-10-07T23:21:21.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.678+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d200bf9e0 con 0x7f2d200cba50 2023-10-07T23:21:21.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.678+0000 7f2d0e7fc700 1 -- 172.21.15.57:0/1360219932 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f2d1002a090 con 0x7f2d200cba50 2023-10-07T23:21:21.685 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:21.686 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 msgr2=0x7f2d00062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 0x7f2d00062a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003aa0 tx=0x7f2d14000bc0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 msgr2=0x7f2d20171680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d20067e20 tx=0x7f2d10017000 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 shutdown_connections 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d200cba50 0x7f2d20171680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2d200c6c10 0x7f2d20171140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2d00060550 0x7f2d00062a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 --2- 172.21.15.57:0/1360219932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2d200bee60 0x7f2d200af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 >> 172.21.15.57:0/1360219932 conn(0x7f2d200670e0 msgr2=0x7f2d200c6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 shutdown_connections 2023-10-07T23:21:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.682+0000 7f2d24f13700 1 -- 172.21.15.57:0/1360219932 wait complete. 2023-10-07T23:21:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2023-10-07T23:21:21.700 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:21.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.798+0000 7ff913fff700 1 Processor -- start 2023-10-07T23:21:21.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.798+0000 7ff913fff700 1 -- start start 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff914156440 0x7ff91406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff91406cd40 0x7ff91406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91406acc0 con 0x7ff9140cadd0 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140c7ed0 con 0x7ff914156440 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91406d1c0 con 0x7ff91406cd40 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff914156440 0x7ff91406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff914156440 0x7ff91406c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42820/0 (socket says 172.21.15.57:42820) 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 -- 172.21.15.57:0/4263126091 learned_addr learned my addr 172.21.15.57:0/4263126091 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:21.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9137fe700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff91406cd40 0x7ff91406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 -- 172.21.15.57:0/4263126091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff91406cd40 msgr2=0x7ff91406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff91406cd40 0x7ff91406b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 -- 172.21.15.57:0/4263126091 >> v1:172.21.15.154:6789/0 conn(0x7ff9140cadd0 legacy=0x7ff9140c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 -- 172.21.15.57:0/4263126091 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140c80f0 con 0x7ff914156440 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff9127fc700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff914156440 0x7ff91406c800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9080099e0 tx=0x7ff908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c3dcd98ca482d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff911ffb700 1 -- 172.21.15.57:0/4263126091 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800b820 con 0x7ff914156440 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff911ffb700 1 -- 172.21.15.57:0/4263126091 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90800b980 con 0x7ff914156440 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff911ffb700 1 -- 172.21.15.57:0/4263126091 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800bd20 con 0x7ff914156440 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- 172.21.15.57:0/4263126091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff914156440 msgr2=0x7ff91406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff914156440 0x7ff91406c800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9080099e0 tx=0x7ff908009250 comp rx=0 tx=0).stop 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- 172.21.15.57:0/4263126091 shutdown_connections 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff91406cd40 0x7ff91406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 --2- 172.21.15.57:0/4263126091 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff914156440 0x7ff91406c800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- 172.21.15.57:0/4263126091 >> 172.21.15.57:0/4263126091 conn(0x7ff9140670e0 msgr2=0x7ff9140674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- 172.21.15.57:0/4263126091 shutdown_connections 2023-10-07T23:21:21.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.802+0000 7ff913fff700 1 -- 172.21.15.57:0/4263126091 wait complete. 2023-10-07T23:21:21.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 Processor -- start 2023-10-07T23:21:21.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- start start 2023-10-07T23:21:21.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9140cadd0 0x7ff914143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9141487e0 0x7ff9141442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914144810 con 0x7ff91406cd40 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff914146bd0 con 0x7ff9141487e0 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141435a0 con 0x7ff9140cadd0 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42826/0 (socket says 172.21.15.57:42826) 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 -- 172.21.15.57:0/1032603938 learned_addr learned my addr 172.21.15.57:0/1032603938 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff9127fc700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9140cadd0 0x7ff914143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff9137fe700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9141487e0 0x7ff9141442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 -- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9140cadd0 msgr2=0x7ff914143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9140cadd0 0x7ff914143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 -- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9141487e0 msgr2=0x7ff9141442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9141487e0 0x7ff9141442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 -- 172.21.15.57:0/1032603938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080094f0 con 0x7ff91406cd40 2023-10-07T23:21:21.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff912ffd700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c790 tx=0x7ff904007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904011590 con 0x7ff91406cd40 2023-10-07T23:21:21.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff904011710 con 0x7ff91406cd40 2023-10-07T23:21:21.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff914146e00 con 0x7ff91406cd40 2023-10-07T23:21:21.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff904011ad0 con 0x7ff91406cd40 2023-10-07T23:21:21.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.806+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9141473c0 con 0x7ff91406cd40 2023-10-07T23:21:21.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.810+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9140711c0 con 0x7ff91406cd40 2023-10-07T23:21:21.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.810+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff90401b3b0 con 0x7ff91406cd40 2023-10-07T23:21:21.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.810+0000 7ff8faffd700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 0x7ff900062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:21.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.810+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff90407b920 con 0x7ff91406cd40 2023-10-07T23:21:21.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.810+0000 7ff9127fc700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 0x7ff900062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:21.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.814+0000 7ff9127fc700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 0x7ff900062a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9080099e0 tx=0x7ff908017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:21.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.818+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff90404c260 con 0x7ff91406cd40 2023-10-07T23:21:21.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.978+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9140c3a60 con 0x7ff91406cd40 2023-10-07T23:21:21.986 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff8faffd700 1 -- 172.21.15.57:0/1032603938 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12127 (secure 0 0 0) 0x7ff904017020 con 0x7ff91406cd40 2023-10-07T23:21:21.986 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:21.986 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:15.837204+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:21.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 msgr2=0x7ff900062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 0x7ff900062a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff9080099e0 tx=0x7ff908017000 comp rx=0 tx=0).stop 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 msgr2=0x7ff9141482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c790 tx=0x7ff904007f40 comp rx=0 tx=0).stop 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 shutdown_connections 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff900060580 0x7ff900062a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9141487e0 0x7ff9141442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff9140cadd0 0x7ff914143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 --2- 172.21.15.57:0/1032603938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff91406cd40 0x7ff9141482a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:21.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 >> 172.21.15.57:0/1032603938 conn(0x7ff9140670e0 msgr2=0x7ff9140ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:21.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 shutdown_connections 2023-10-07T23:21:21.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:21.982+0000 7ff913fff700 1 -- 172.21.15.57:0/1032603938 wait complete. 2023-10-07T23:21:22.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:21:22.007 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:22.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.114+0000 7fcc89056700 1 Processor -- start 2023-10-07T23:21:22.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.114+0000 7fcc89056700 1 -- start start 2023-10-07T23:21:22.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcc841596f0 0x7fcc840cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc840c9580 0x7fcc840c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc84159c60 con 0x7fcc84155610 2023-10-07T23:21:22.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc84159f40 con 0x7fcc841596f0 2023-10-07T23:21:22.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415a220 con 0x7fcc840c9580 2023-10-07T23:21:22.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc83fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcc84155610 0x7fcc841559e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54588/0 (socket says 172.21.15.57:54588) 2023-10-07T23:21:22.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc83fff700 1 -- 172.21.15.57:0/3635834032 learned_addr learned my addr 172.21.15.57:0/3635834032 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:22.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 --2- 172.21.15.57:0/3635834032 >> v2:172.21.15.57:3300/0 conn(0x7fcc841596f0 0x7fcc840cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc88855700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc840c9580 0x7fcc840c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627987740 0 0) 0x7fcc84159c60 con 0x7fcc84155610 2023-10-07T23:21:22.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6c003570 con 0x7fcc84155610 2023-10-07T23:21:22.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 -- 172.21.15.57:0/3635834032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc840c9580 msgr2=0x7fcc840c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:22.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc840c9580 0x7fcc840c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201048522 0 0) 0x7fcc6c003570 con 0x7fcc84155610 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 -- 172.21.15.57:0/3635834032 >> v1:172.21.15.57:6789/0 conn(0x7fcc84155610 legacy=0x7fcc841559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 -- 172.21.15.57:0/3635834032 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8415a500 con 0x7fcc841596f0 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc837fe700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc841596f0 0x7fcc840cb080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc780098e0 tx=0x7fcc780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf00bcab96c55ed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74001470 con 0x7fcc841596f0 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78011420 con 0x7fcc841596f0 2023-10-07T23:21:22.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc82ffd700 1 -- 172.21.15.57:0/3635834032 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc780117a0 con 0x7fcc841596f0 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- 172.21.15.57:0/3635834032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc841596f0 msgr2=0x7fcc840cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc841596f0 0x7fcc840cb080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc780098e0 tx=0x7fcc780091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- 172.21.15.57:0/3635834032 shutdown_connections 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc840c9580 0x7fcc840c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 --2- 172.21.15.57:0/3635834032 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc841596f0 0x7fcc840cb080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- 172.21.15.57:0/3635834032 >> 172.21.15.57:0/3635834032 conn(0x7fcc840670e0 msgr2=0x7fcc840674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- 172.21.15.57:0/3635834032 shutdown_connections 2023-10-07T23:21:22.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.118+0000 7fcc89056700 1 -- 172.21.15.57:0/3635834032 wait complete. 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 Processor -- start 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- start start 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 0x7fcc840c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc841596f0 0x7fcc840c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84169780 con 0x7fcc84155610 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84169a60 con 0x7fcc840c9580 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84169d40 con 0x7fcc841596f0 2023-10-07T23:21:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42840/0 (socket says 172.21.15.57:42840) 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 -- 172.21.15.57:0/4000648049 learned_addr learned my addr 172.21.15.57:0/4000648049 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc88855700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc841596f0 0x7fcc840c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc83fff700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 0x7fcc840c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 -- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc841596f0 msgr2=0x7fcc840c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc841596f0 0x7fcc840c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 -- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 msgr2=0x7fcc840c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 0x7fcc840c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 -- 172.21.15.57:0/4000648049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78009420 con 0x7fcc84155610 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc83fff700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 0x7fcc840c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc837fe700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc84068610 tx=0x7fcc78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78021070 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7801c2e0 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8416a020 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78000ce0 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.122+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc840b18f0 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc840711c0 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcc7801a560 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc6bfff700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 0x7fcc70062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcc7807f220 con 0x7fcc84155610 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc83fff700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 0x7fcc70062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.126+0000 7fcc83fff700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 0x7fcc70062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c00 tx=0x7fcc740011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.134+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcc7804fb60 con 0x7fcc84155610 2023-10-07T23:21:22.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.290+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc840ca630 con 0x7fcc84155610 2023-10-07T23:21:22.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc6bfff700 1 -- 172.21.15.57:0/4000648049 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc78053180 con 0x7fcc84155610 2023-10-07T23:21:22.298 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 msgr2=0x7fcc70062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 0x7fcc70062a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc74000c00 tx=0x7fcc740011a0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 msgr2=0x7fcc840c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc84068610 tx=0x7fcc78016000 comp rx=0 tx=0).stop 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 shutdown_connections 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcc841596f0 0x7fcc840c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc84155610 0x7fcc840c4a80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcc70060550 0x7fcc70062a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 --2- 172.21.15.57:0/4000648049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc840c9580 0x7fcc840c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 >> 172.21.15.57:0/4000648049 conn(0x7fcc840670e0 msgr2=0x7fcc84154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:22.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 shutdown_connections 2023-10-07T23:21:22.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.294+0000 7fcc89056700 1 -- 172.21.15.57:0/4000648049 wait complete. 2023-10-07T23:21:22.312 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T23:21:22.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 Processor -- start 2023-10-07T23:21:22.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 -- start start 2023-10-07T23:21:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f48181403c0 0x7f48180cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c7100 0x7f481806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180cafc0 con 0x7f4818140790 2023-10-07T23:21:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180c8090 con 0x7f48181403c0 2023-10-07T23:21:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481e617700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180cc7d0 con 0x7f48180c7100 2023-10-07T23:21:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481d615700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f48181403c0 0x7f48180cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.414+0000 7f481d615700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f48181403c0 0x7f48180cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42846/0 (socket says 172.21.15.57:42846) 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 -- 172.21.15.57:0/406620326 learned_addr learned my addr 172.21.15.57:0/406620326 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481de16700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c7100 0x7f481806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 -- 172.21.15.57:0/406620326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c7100 msgr2=0x7f481806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c7100 0x7f481806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 -- 172.21.15.57:0/406620326 >> v1:172.21.15.154:6789/0 conn(0x7f4818140790 legacy=0x7f48180cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f480ffff700 1 -- 172.21.15.57:0/406620326 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222559154 0 0) 0x7f48180cafc0 con 0x7f4818140790 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 -- 172.21.15.57:0/406620326 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180ca690 con 0x7f48181403c0 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481d615700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f48180cb450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48140099b0 tx=0x7f4814009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24272a74fe0ccd34 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f480ffff700 1 -- 172.21.15.57:0/406620326 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814009ea0 con 0x7f48181403c0 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f480ffff700 1 -- 172.21.15.57:0/406620326 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481400b860 con 0x7f48181403c0 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f480ffff700 1 -- 172.21.15.57:0/406620326 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481400bc00 con 0x7f48181403c0 2023-10-07T23:21:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 -- 172.21.15.57:0/406620326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 msgr2=0x7f48180cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f48180cb450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f48140099b0 tx=0x7f4814009250 comp rx=0 tx=0).stop 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 -- 172.21.15.57:0/406620326 shutdown_connections 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c7100 0x7f481806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 --2- 172.21.15.57:0/406620326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f48180cb450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 -- 172.21.15.57:0/406620326 >> 172.21.15.57:0/406620326 conn(0x7f4818095d30 msgr2=0x7f481806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 -- 172.21.15.57:0/406620326 shutdown_connections 2023-10-07T23:21:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.418+0000 7f481e617700 1 -- 172.21.15.57:0/406620326 wait complete. 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 Processor -- start 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- start start 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 0x7f48180a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4818140790 0x7f481813e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180aa780 con 0x7f48181403c0 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180aa1c0 con 0x7f48180c7100 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180aa4a0 con 0x7f4818140790 2023-10-07T23:21:22.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42852/0 (socket says 172.21.15.57:42852) 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 -- 172.21.15.57:0/4090199400 learned_addr learned my addr 172.21.15.57:0/4090199400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481de16700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4818140790 0x7f481813e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481d615700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 0x7f48180a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 -- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4818140790 msgr2=0x7f481813e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4818140790 0x7f481813e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 -- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 msgr2=0x7f48180a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 0x7f48180a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 -- 172.21.15.57:0/4090199400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140094f0 con 0x7f48181403c0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481d615700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 0x7f48180a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481ce14700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f480800c530 tx=0x7f480800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808011590 con 0x7f48181403c0 2023-10-07T23:21:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808011710 con 0x7f48181403c0 2023-10-07T23:21:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48180986f0 con 0x7f48181403c0 2023-10-07T23:21:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808011ad0 con 0x7f48181403c0 2023-10-07T23:21:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.422+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48180a9b70 con 0x7f48181403c0 2023-10-07T23:21:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818098bf0 con 0x7f48181403c0 2023-10-07T23:21:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f480801b3b0 con 0x7f48181403c0 2023-10-07T23:21:22.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f480d7fa700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 0x7f47fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f480807ba20 con 0x7f48181403c0 2023-10-07T23:21:22.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f481d615700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 0x7f47fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.426+0000 7f481d615700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 0x7f47fc062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48140099b0 tx=0x7f4814017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.438+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f480804c2c0 con 0x7f48181403c0 2023-10-07T23:21:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.598+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f48180711c0 con 0x7f48181403c0 2023-10-07T23:21:22.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f480d7fa700 1 -- 172.21.15.57:0/4090199400 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4808017020 con 0x7f48181403c0 2023-10-07T23:21:22.606 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:22.606 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-07T23:21:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:21:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-10-07T23:21:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-10-07T23:21:22.607 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:21:22.607 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-07T23:21:22.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 msgr2=0x7f47fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 0x7f47fc062a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f48140099b0 tx=0x7f4814017000 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 msgr2=0x7f481816a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f480800c530 tx=0x7f480800c840 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 shutdown_connections 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4818140790 0x7f481813e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48181403c0 0x7f481816a480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f47fc060550 0x7f47fc062a00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 --2- 172.21.15.57:0/4090199400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180c7100 0x7f48180a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 >> 172.21.15.57:0/4090199400 conn(0x7f4818095d30 msgr2=0x7f48180afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 shutdown_connections 2023-10-07T23:21:22.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.602+0000 7f481e617700 1 -- 172.21.15.57:0/4090199400 wait complete. 2023-10-07T23:21:22.621 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T23:21:22.622 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T23:21:22.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.726+0000 7f7675892700 1 Processor -- start 2023-10-07T23:21:22.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.726+0000 7f7675892700 1 -- start start 2023-10-07T23:21:22.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7670159030 0x7f76700aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 0x7f767006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767006a900 con 0x7f76700ca600 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76700b0450 con 0x7f7670159030 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006a600 con 0x7f767006c800 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7670159030 0x7f76700aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7670159030 0x7f76700aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42862/0 (socket says 172.21.15.57:42862) 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 -- 172.21.15.57:0/1499094487 learned_addr learned my addr 172.21.15.57:0/1499094487 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766e7fc700 1 --2- 172.21.15.57:0/1499094487 >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 0x7f767006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 -- 172.21.15.57:0/1499094487 >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 msgr2=0x7f767006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 --2- 172.21.15.57:0/1499094487 >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 0x7f767006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 -- 172.21.15.57:0/1499094487 >> v1:172.21.15.154:6789/0 conn(0x7f76700ca600 legacy=0x7f76700caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 -- 172.21.15.57:0/1499094487 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767006d170 con 0x7f7670159030 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766e7fc700 1 --2- 172.21.15.57:0/1499094487 >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 0x7f767006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:22.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766effd700 1 --2- 172.21.15.57:0/1499094487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7670159030 0x7f76700aff10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76700ca540 tx=0x7f7664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f654660c64a1b3ec server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766dffb700 1 -- 172.21.15.57:0/1499094487 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400b860 con 0x7f7670159030 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766dffb700 1 -- 172.21.15.57:0/1499094487 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400b9c0 con 0x7f7670159030 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f766dffb700 1 -- 172.21.15.57:0/1499094487 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400bd60 con 0x7f7670159030 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- 172.21.15.57:0/1499094487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7670159030 msgr2=0x7f76700aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 --2- 172.21.15.57:0/1499094487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7670159030 0x7f76700aff10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f76700ca540 tx=0x7f7664009250 comp rx=0 tx=0).stop 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- 172.21.15.57:0/1499094487 shutdown_connections 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 --2- 172.21.15.57:0/1499094487 >> v2:172.21.15.154:3300/0 conn(0x7f767006c800 0x7f767006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 --2- 172.21.15.57:0/1499094487 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7670159030 0x7f76700aff10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- 172.21.15.57:0/1499094487 >> 172.21.15.57:0/1499094487 conn(0x7f7670068650 msgr2=0x7f7670068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:22.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- 172.21.15.57:0/1499094487 shutdown_connections 2023-10-07T23:21:22.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.730+0000 7f7675892700 1 -- 172.21.15.57:0/1499094487 wait complete. 2023-10-07T23:21:22.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.734+0000 7f7675892700 1 Processor -- start 2023-10-07T23:21:22.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.734+0000 7f7675892700 1 -- start start 2023-10-07T23:21:22.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 0x7f767016a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7670159030 0x7f767016c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700c8580 con 0x7f767006c800 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700c6680 con 0x7f76700ca600 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700c8820 con 0x7f7670159030 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42878/0 (socket says 172.21.15.57:42878) 2023-10-07T23:21:22.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 -- 172.21.15.57:0/3304493093 learned_addr learned my addr 172.21.15.57:0/3304493093 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766e7fc700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 0x7f767016a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766f7fe700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7670159030 0x7f767016c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 -- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7670159030 msgr2=0x7f767016c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7670159030 0x7f767016c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 -- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 msgr2=0x7f767016a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:22.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 0x7f767016a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 -- 172.21.15.57:0/3304493093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640094f0 con 0x7f767006c800 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766e7fc700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 0x7f767016a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f766effd700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7664009290 tx=0x7f7664017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664011e60 con 0x7f767006c800 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766401f070 con 0x7f767006c800 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76700c8b00 con 0x7f767006c800 2023-10-07T23:21:22.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766401a8b0 con 0x7f767006c800 2023-10-07T23:21:22.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76700c6950 con 0x7f767006c800 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.738+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76700647b0 con 0x7f767006c800 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.742+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f766401ac20 con 0x7f767006c800 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.742+0000 7f7656ffd700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 0x7f765c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.742+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7664021070 con 0x7f767006c800 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.742+0000 7f766e7fc700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 0x7f765c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:22.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.742+0000 7f766e7fc700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 0x7f765c062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7660006f00 tx=0x7f7660008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:22.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.750+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f766404ed80 con 0x7f767006c800 2023-10-07T23:21:22.916 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:22.910+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76700cd5e0 con 0x7f767006c800 2023-10-07T23:21:23.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.918+0000 7f7656ffd700 1 -- 172.21.15.57:0/3304493093 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f766402b020 con 0x7f767006c800 2023-10-07T23:21:23.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 msgr2=0x7f765c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:23.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 0x7f765c062a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7660006f00 tx=0x7f7660008040 comp rx=0 tx=0).stop 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 msgr2=0x7f76700c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7664009290 tx=0x7f7664017000 comp rx=0 tx=0).stop 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 shutdown_connections 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7670159030 0x7f767016c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f765c060550 0x7f765c062a00 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7660006f00 tx=0x7f7660008040 comp rx=0 tx=0).stop 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76700ca600 0x7f767016a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:23.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 --2- 172.21.15.57:0/3304493093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f767006c800 0x7f76700c6140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:23.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.926+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 >> 172.21.15.57:0/3304493093 conn(0x7f7670068650 msgr2=0x7f76700cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:23.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.930+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 shutdown_connections 2023-10-07T23:21:23.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:23.930+0000 7f7675892700 1 -- 172.21.15.57:0/3304493093 wait complete. 2023-10-07T23:21:23.936 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-10-07T23:21:23.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T23:21:24.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.054+0000 7f2b57104700 1 Processor -- start 2023-10-07T23:21:24.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.054+0000 7f2b57104700 1 -- start start 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b57104700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b500cad10 0x7f2b500aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b57104700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5006a900 con 0x7f2b5006cf10 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b57104700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5006a600 con 0x7f2b50159030 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b57104700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b500b0450 con 0x7f2b500cad10 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4ffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f2b5006cf10 0x7f2b500ca600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54600/0 (socket says 172.21.15.57:54600) 2023-10-07T23:21:24.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4ffff700 1 -- 172.21.15.57:0/323883686 learned_addr learned my addr 172.21.15.57:0/323883686 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 --2- 172.21.15.57:0/323883686 >> v2:172.21.15.154:3300/0 conn(0x7f2b500cad10 0x7f2b500aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162783587 0 0) 0x7f2b5006a900 con 0x7f2b5006cf10 2023-10-07T23:21:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b40003570 con 0x7f2b5006cf10 2023-10-07T23:21:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135208901 0 0) 0x7f2b5006a600 con 0x7f2b50159030 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b5006a900 con 0x7f2b50159030 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3221507718 0 0) 0x7f2b40003570 con 0x7f2b5006cf10 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 >> v2:172.21.15.154:3300/0 conn(0x7f2b500cad10 msgr2=0x7f2b500aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 --2- 172.21.15.57:0/323883686 >> v2:172.21.15.154:3300/0 conn(0x7f2b500cad10 0x7f2b500aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 >> v1:172.21.15.154:6789/0 conn(0x7f2b50159030 legacy=0x7f2b5006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500cb350 con 0x7f2b5006cf10 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f2b3c001470 con 0x7f2b5006cf10 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 >> v1:172.21.15.57:6789/0 conn(0x7f2b5006cf10 legacy=0x7f2b500ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b40003bf0 0x7f2b40003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b40004810 0x7f2b40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b0450 con 0x7f2b400051c0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4000c0b0 con 0x7f2b40004810 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4000c270 con 0x7f2b40003bf0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b54ea0700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b40003bf0 0x7f2b40003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4ffff700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b40004810 0x7f2b40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 -- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b40003bf0 msgr2=0x7f2b40003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b40003bf0 0x7f2b40003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 -- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b40004810 msgr2=0x7f2b40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b40004810 0x7f2b40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 -- 172.21.15.57:0/323883686 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40003570 con 0x7f2b400051c0 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b4ffff700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b40004810 0x7f2b40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 -- 172.21.15.57:0/323883686 reap_dead start 2023-10-07T23:21:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.058+0000 7f2b556a1700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b4000ba60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e210 tx=0x7f2b4800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3653438b4fa2a3d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800eac0 con 0x7f2b400051c0 2023-10-07T23:21:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4800f040 con 0x7f2b400051c0 2023-10-07T23:21:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b4f7fe700 1 -- 172.21.15.57:0/323883686 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800ee70 con 0x7f2b400051c0 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 msgr2=0x7f2b4000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b4000ba60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800e210 tx=0x7f2b4800b040 comp rx=0 tx=0).stop 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- 172.21.15.57:0/323883686 shutdown_connections 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 --2- 172.21.15.57:0/323883686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b4000ba60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- 172.21.15.57:0/323883686 >> 172.21.15.57:0/323883686 conn(0x7f2b50068650 msgr2=0x7f2b50068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- 172.21.15.57:0/323883686 shutdown_connections 2023-10-07T23:21:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- 172.21.15.57:0/323883686 wait complete. 2023-10-07T23:21:24.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 Processor -- start 2023-10-07T23:21:24.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.062+0000 7f2b57104700 1 -- start start 2023-10-07T23:21:24.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 0x7f2b500cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 0x7f2b500cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5015ae90 con 0x7f2b400051c0 2023-10-07T23:21:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b50154a80 con 0x7f2b5015b790 2023-10-07T23:21:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500c5960 con 0x7f2b5015b320 2023-10-07T23:21:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b54ea0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b54ea0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42894/0 (socket says 172.21.15.57:42894) 2023-10-07T23:21:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b54ea0700 1 -- 172.21.15.57:0/864435894 learned_addr learned my addr 172.21.15.57:0/864435894 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b4ffff700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 0x7f2b500cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 0x7f2b500cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 -- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 msgr2=0x7f2b500cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 0x7f2b500cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 -- 172.21.15.57:0/864435894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 msgr2=0x7f2b50154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 -- 172.21.15.57:0/864435894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5015a840 con 0x7f2b5015b790 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b54ea0700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b4ffff700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 0x7f2b500cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b556a1700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 0x7f2b500cf4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006a050 tx=0x7f2b48019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:24.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801d660 con 0x7f2b5015b790 2023-10-07T23:21:24.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48022070 con 0x7f2b5015b790 2023-10-07T23:21:24.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801da30 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5015ab20 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.066+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5015a600 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b500647b0 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2b48003020 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b4cff9700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 0x7f2b34062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2b48024070 con 0x7f2b5015b790 2023-10-07T23:21:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b54ea0700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 0x7f2b34062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.070+0000 7f2b54ea0700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 0x7f2b34062a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b440010e0 tx=0x7f2b44007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:24.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.082+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2b4804f170 con 0x7f2b5015b790 2023-10-07T23:21:24.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.246+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b50068e30 con 0x7f2b5015b790 2023-10-07T23:21:24.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.934+0000 7f2b4cff9700 1 -- 172.21.15.57:0/864435894 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b48052710 con 0x7f2b5015b790 2023-10-07T23:21:24.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 msgr2=0x7f2b34062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 0x7f2b34062a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b440010e0 tx=0x7f2b44007d30 comp rx=0 tx=0).stop 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 msgr2=0x7f2b500cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 0x7f2b500cf4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006a050 tx=0x7f2b48019000 comp rx=0 tx=0).stop 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 shutdown_connections 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2b34060580 0x7f2b34062a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b5015b790 0x7f2b500cf4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2b5015b320 0x7f2b500cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 --2- 172.21.15.57:0/864435894 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2b400051c0 0x7f2b50154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 >> 172.21.15.57:0/864435894 conn(0x7f2b50068650 msgr2=0x7f2b500b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 shutdown_connections 2023-10-07T23:21:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:24.958+0000 7f2b57104700 1 -- 172.21.15.57:0/864435894 wait complete. 2023-10-07T23:21:24.964 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-10-07T23:21:24.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T23:21:25.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 Processor -- start 2023-10-07T23:21:25.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 -- start start 2023-10-07T23:21:25.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5f380add20 0x7f5f380ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5f380abdb0 0x7f5f380ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380c75b0 con 0x7f5f38156760 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380c69e0 con 0x7f5f380add20 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.082+0000 7f5f3759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380c6c80 con 0x7f5f380abdb0 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f36d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5f38156760 0x7f5f38156ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54616/0 (socket says 172.21.15.57:54616) 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5f380add20 0x7f5f380ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f36d9d700 1 -- 172.21.15.57:0/2721772200 learned_addr learned my addr 172.21.15.57:0/2721772200 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f2ffff700 1 --2- 172.21.15.57:0/2721772200 >> v2:172.21.15.154:3300/0 conn(0x7f5f380abdb0 0x7f5f380ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 -- 172.21.15.57:0/2721772200 >> v2:172.21.15.154:3300/0 conn(0x7f5f380abdb0 msgr2=0x7f5f380ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 --2- 172.21.15.57:0/2721772200 >> v2:172.21.15.154:3300/0 conn(0x7f5f380abdb0 0x7f5f380ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 -- 172.21.15.57:0/2721772200 >> v1:172.21.15.57:6789/0 conn(0x7f5f38156760 legacy=0x7f5f38156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 -- 172.21.15.57:0/2721772200 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380c6f60 con 0x7f5f380add20 2023-10-07T23:21:25.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3659c700 1 --2- 172.21.15.57:0/2721772200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380add20 0x7f5f380ae0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f380689e0 tx=0x7f5f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f6e37023c15eca server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f35d9b700 1 -- 172.21.15.57:0/2721772200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800b6b0 con 0x7f5f380add20 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f35d9b700 1 -- 172.21.15.57:0/2721772200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2800b810 con 0x7f5f380add20 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f35d9b700 1 -- 172.21.15.57:0/2721772200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800bbb0 con 0x7f5f380add20 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- 172.21.15.57:0/2721772200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380add20 msgr2=0x7f5f380ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 --2- 172.21.15.57:0/2721772200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380add20 0x7f5f380ae0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f380689e0 tx=0x7f5f280091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- 172.21.15.57:0/2721772200 shutdown_connections 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 --2- 172.21.15.57:0/2721772200 >> v2:172.21.15.154:3300/0 conn(0x7f5f380abdb0 0x7f5f380ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 --2- 172.21.15.57:0/2721772200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380add20 0x7f5f380ae0f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- 172.21.15.57:0/2721772200 >> 172.21.15.57:0/2721772200 conn(0x7f5f380670e0 msgr2=0x7f5f380674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- 172.21.15.57:0/2721772200 shutdown_connections 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- 172.21.15.57:0/2721772200 wait complete. 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 Processor -- start 2023-10-07T23:21:25.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.086+0000 7f5f3759e700 1 -- start start 2023-10-07T23:21:25.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f380add20 0x7f5f38152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f38156760 0x7f5f380c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380bfa80 con 0x7f5f380abdb0 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381532d0 con 0x7f5f380add20 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c55a0 con 0x7f5f38156760 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f380add20 0x7f5f38152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42912/0 (socket says 172.21.15.57:42912) 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f36d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f38156760 0x7f5f380c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 -- 172.21.15.57:0/1837534005 learned_addr learned my addr 172.21.15.57:0/1837534005 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:25.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 -- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f38156760 msgr2=0x7f5f380c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f38156760 0x7f5f380c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 -- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f380add20 msgr2=0x7f5f38152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f380add20 0x7f5f38152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 -- 172.21.15.57:0/1837534005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009420 con 0x7f5f380abdb0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3659c700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f3806c620 tx=0x7f5f28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2800bb90 con 0x7f5f380abdb0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28011420 con 0x7f5f380abdb0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f380c5880 con 0x7f5f380abdb0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280117c0 con 0x7f5f380abdb0 2023-10-07T23:21:25.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.090+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380bf710 con 0x7f5f380abdb0 2023-10-07T23:21:25.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380711c0 con 0x7f5f380abdb0 2023-10-07T23:21:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5f28011bb0 con 0x7f5f380abdb0 2023-10-07T23:21:25.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f2e7fc700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 0x7f5f18062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:25.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f5f2807dff0 con 0x7f5f380abdb0 2023-10-07T23:21:25.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f2ffff700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 0x7f5f18062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:25.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.094+0000 7f5f2ffff700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 0x7f5f18062a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f380bf290 tx=0x7f5f20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:25.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.102+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5f2804e5a0 con 0x7f5f380abdb0 2023-10-07T23:21:25.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.302+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5f380c6130 con 0x7f5f380abdb0 2023-10-07T23:21:25.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.938+0000 7f5f2e7fc700 1 -- 172.21.15.57:0/1837534005 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5f28011e60 con 0x7f5f380abdb0 2023-10-07T23:21:25.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.942+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 msgr2=0x7f5f18062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.942+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 0x7f5f18062a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f380bf290 tx=0x7f5f20008040 comp rx=0 tx=0).stop 2023-10-07T23:21:25.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.942+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 msgr2=0x7f5f380c1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:25.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.942+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5f3806c620 tx=0x7f5f28017000 comp rx=0 tx=0).stop 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 shutdown_connections 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f38156760 0x7f5f380c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f18060580 0x7f5f18062a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f380add20 0x7f5f38152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 --2- 172.21.15.57:0/1837534005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f380abdb0 0x7f5f380c1f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 >> 172.21.15.57:0/1837534005 conn(0x7f5f380670e0 msgr2=0x7f5f38153720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 shutdown_connections 2023-10-07T23:21:25.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:25.950+0000 7f5f3759e700 1 -- 172.21.15.57:0/1837534005 wait complete. 2023-10-07T23:21:25.957 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 2 and data pool 3 2023-10-07T23:21:25.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.070+0000 7f4e377f8700 1 Processor -- start 2023-10-07T23:21:26.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.070+0000 7f4e377f8700 1 -- start start 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e3006cf10 0x7f4e301599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4e300af380 0x7f4e30159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3015ccb0 con 0x7f4e300aefb0 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3015cf70 con 0x7f4e3006cf10 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3015d250 con 0x7f4e300af380 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4e3006cf10 0x7f4e301599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e35594700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4e300aefb0 0x7f4e3006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54624/0 (socket says 172.21.15.57:54624) 2023-10-07T23:21:26.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e35594700 1 -- 172.21.15.57:0/3710211208 learned_addr learned my addr 172.21.15.57:0/3710211208 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e35d95700 1 --2- 172.21.15.57:0/3710211208 >> v2:172.21.15.154:3300/0 conn(0x7f4e300af380 0x7f4e30159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 -- 172.21.15.57:0/3710211208 >> v2:172.21.15.154:3300/0 conn(0x7f4e300af380 msgr2=0x7f4e30159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 --2- 172.21.15.57:0/3710211208 >> v2:172.21.15.154:3300/0 conn(0x7f4e300af380 0x7f4e30159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 -- 172.21.15.57:0/3710211208 >> v1:172.21.15.57:6789/0 conn(0x7f4e300aefb0 legacy=0x7f4e3006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 -- 172.21.15.57:0/3710211208 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3015d530 con 0x7f4e3006cf10 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e27fff700 1 -- 172.21.15.57:0/3710211208 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168861606 0 0) 0x7f4e3015ccb0 con 0x7f4e300aefb0 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e34d93700 1 --2- 172.21.15.57:0/3710211208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3006cf10 0x7f4e301599e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e300ca5b0 tx=0x7f4e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64904818e6c5a1a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e27fff700 1 -- 172.21.15.57:0/3710211208 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c011630 con 0x7f4e3006cf10 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e27fff700 1 -- 172.21.15.57:0/3710211208 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c011790 con 0x7f4e3006cf10 2023-10-07T23:21:26.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e27fff700 1 -- 172.21.15.57:0/3710211208 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c011b30 con 0x7f4e3006cf10 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- 172.21.15.57:0/3710211208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3006cf10 msgr2=0x7f4e301599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3710211208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3006cf10 0x7f4e301599e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e300ca5b0 tx=0x7f4e2c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- 172.21.15.57:0/3710211208 shutdown_connections 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3710211208 >> v2:172.21.15.154:3300/0 conn(0x7f4e300af380 0x7f4e30159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3710211208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e3006cf10 0x7f4e301599e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- 172.21.15.57:0/3710211208 >> 172.21.15.57:0/3710211208 conn(0x7f4e30068630 msgr2=0x7f4e30068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- 172.21.15.57:0/3710211208 shutdown_connections 2023-10-07T23:21:26.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.074+0000 7f4e377f8700 1 -- 172.21.15.57:0/3710211208 wait complete. 2023-10-07T23:21:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 Processor -- start 2023-10-07T23:21:26.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 -- start start 2023-10-07T23:21:26.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e3006cf10 0x7f4e30156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e300aefb0 0x7f4e300b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.078+0000 7f4e377f8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e30157330 con 0x7f4e300af380 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e377f8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b0250 con 0x7f4e300aefb0 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e377f8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b3d50 con 0x7f4e3006cf10 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42930/0 (socket says 172.21.15.57:42930) 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 -- 172.21.15.57:0/3600063536 learned_addr learned my addr 172.21.15.57:0/3600063536 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35594700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e3006cf10 0x7f4e30156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e34d93700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e300aefb0 0x7f4e300b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 -- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e3006cf10 msgr2=0x7f4e30156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e3006cf10 0x7f4e30156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 -- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e300aefb0 msgr2=0x7f4e300b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e300aefb0 0x7f4e300b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 -- 172.21.15.57:0/3600063536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c009420 con 0x7f4e300af380 2023-10-07T23:21:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e35d95700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009df0 tx=0x7f4e2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800caa0 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2800f040 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e300b2780 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800ce70 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e300b04a0 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4e28014690 con 0x7f4e300af380 2023-10-07T23:21:26.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.082+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e300711c0 con 0x7f4e300af380 2023-10-07T23:21:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.086+0000 7f4e257fa700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 0x7f4e14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.086+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e2807cfa0 con 0x7f4e300af380 2023-10-07T23:21:26.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.086+0000 7f4e35594700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 0x7f4e14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.086+0000 7f4e35594700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 0x7f4e14062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007fd0 tx=0x7f4e20007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.094+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4e280acbe0 con 0x7f4e300af380 2023-10-07T23:21:26.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.258+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e3015d810 con 0x7f4e300af380 2023-10-07T23:21:26.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e257fa700 1 -- 172.21.15.57:0/3600063536 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f4e28018090 con 0x7f4e300af380 2023-10-07T23:21:26.266 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:26.266 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:25.307677+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:26.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 msgr2=0x7f4e14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 0x7f4e14062a00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007fd0 tx=0x7f4e20007d30 comp rx=0 tx=0).stop 2023-10-07T23:21:26.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 msgr2=0x7f4e300b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009df0 tx=0x7f4e2800b040 comp rx=0 tx=0).stop 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 shutdown_connections 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4e300af380 0x7f4e300b2240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4e14060550 0x7f4e14062a00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e300aefb0 0x7f4e300b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 --2- 172.21.15.57:0/3600063536 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4e3006cf10 0x7f4e30156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 >> 172.21.15.57:0/3600063536 conn(0x7f4e30068630 msgr2=0x7f4e300cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 shutdown_connections 2023-10-07T23:21:26.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.262+0000 7f4e377f8700 1 -- 172.21.15.57:0/3600063536 wait complete. 2023-10-07T23:21:26.281 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:26.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 Processor -- start 2023-10-07T23:21:26.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 -- start start 2023-10-07T23:21:26.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f7518069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f751806c800 0x7f751806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751815aae0 con 0x7f75180abf00 2023-10-07T23:21:26.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751806d1c0 con 0x7f751806c800 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f751c808700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75180ac2d0 con 0x7f75180c75e0 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f7516d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f75180abf00 0x7f751815af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:54632/0 (socket says 172.21.15.57:54632) 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.382+0000 7f7516d9d700 1 -- 172.21.15.57:0/406376442 learned_addr learned my addr 172.21.15.57:0/406376442 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 --2- 172.21.15.57:0/406376442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f7518069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751759e700 1 --2- 172.21.15.57:0/406376442 >> v2:172.21.15.154:3300/0 conn(0x7f751806c800 0x7f751806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f7515d9b700 1 -- 172.21.15.57:0/406376442 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667879679 0 0) 0x7f751815aae0 con 0x7f75180abf00 2023-10-07T23:21:26.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f7515d9b700 1 -- 172.21.15.57:0/406376442 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7504003570 con 0x7f75180abf00 2023-10-07T23:21:26.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 -- 172.21.15.57:0/406376442 >> v2:172.21.15.154:3300/0 conn(0x7f751806c800 msgr2=0x7f751806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:26.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 --2- 172.21.15.57:0/406376442 >> v2:172.21.15.154:3300/0 conn(0x7f751806c800 0x7f751806cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 -- 172.21.15.57:0/406376442 >> v1:172.21.15.57:6789/0 conn(0x7f75180abf00 legacy=0x7f751815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 -- 172.21.15.57:0/406376442 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518157f90 con 0x7f75180c75e0 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751659c700 1 --2- 172.21.15.57:0/406376442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f7518069c30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75000098e0 tx=0x7f75000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80a70250a6a23aa server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f7515d9b700 1 -- 172.21.15.57:0/406376442 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500011630 con 0x7f75180c75e0 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f7515d9b700 1 -- 172.21.15.57:0/406376442 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7500011790 con 0x7f75180c75e0 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f7515d9b700 1 -- 172.21.15.57:0/406376442 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500011b10 con 0x7f75180c75e0 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 -- 172.21.15.57:0/406376442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 msgr2=0x7f7518069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 --2- 172.21.15.57:0/406376442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f7518069c30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75000098e0 tx=0x7f75000091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 -- 172.21.15.57:0/406376442 shutdown_connections 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 --2- 172.21.15.57:0/406376442 >> v2:172.21.15.154:3300/0 conn(0x7f751806c800 0x7f751806cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 --2- 172.21.15.57:0/406376442 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f7518069c30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 -- 172.21.15.57:0/406376442 >> 172.21.15.57:0/406376442 conn(0x7f751806b5d0 msgr2=0x7f751806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 -- 172.21.15.57:0/406376442 shutdown_connections 2023-10-07T23:21:26.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.386+0000 7f751c808700 1 -- 172.21.15.57:0/406376442 wait complete. 2023-10-07T23:21:26.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 Processor -- start 2023-10-07T23:21:26.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- start start 2023-10-07T23:21:26.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75180abf00 0x7f751814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f751806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75180ad5b0 con 0x7f751806c800 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75180ae330 con 0x7f75180abf00 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751814ebe0 con 0x7f75180c75e0 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42946/0 (socket says 172.21.15.57:42946) 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 -- 172.21.15.57:0/1886930722 learned_addr learned my addr 172.21.15.57:0/1886930722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751659c700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75180abf00 0x7f751814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751759e700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f751806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 -- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 msgr2=0x7f751806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f751806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 -- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75180abf00 msgr2=0x7f751814dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75180abf00 0x7f751814dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 -- 172.21.15.57:0/1886930722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009420 con 0x7f751806c800 2023-10-07T23:21:26.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f7516d9d700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7508002ee0 tx=0x7f7508007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75080072e0 con 0x7f751806c800 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7508007460 con 0x7f751806c800 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75180ada00 con 0x7f751806c800 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75080057f0 con 0x7f751806c800 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.390+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75180adfc0 con 0x7f751806c800 2023-10-07T23:21:26.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.394+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180711c0 con 0x7f751806c800 2023-10-07T23:21:26.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.394+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7508005be0 con 0x7f751806c800 2023-10-07T23:21:26.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.394+0000 7f750effd700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 0x7f74f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.398+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f750807bd40 con 0x7f751806c800 2023-10-07T23:21:26.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.398+0000 7f751659c700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 0x7f74f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.398+0000 7f751659c700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 0x7f74f8062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7518154c00 tx=0x7f7500017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.406+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f750804c330 con 0x7f751806c800 2023-10-07T23:21:26.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.566+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75180c80f0 con 0x7f751806c800 2023-10-07T23:21:26.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.570+0000 7f750effd700 1 -- 172.21.15.57:0/1886930722 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f7508017090 con 0x7f751806c800 2023-10-07T23:21:26.576 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:26.576 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:25.307677+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:26.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 msgr2=0x7f74f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 0x7f74f8062a00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7518154c00 tx=0x7f7500017000 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 msgr2=0x7f75180aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7508002ee0 tx=0x7f7508007c60 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 shutdown_connections 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f75180c75e0 0x7f751806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f74f8060550 0x7f74f8062a00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75180abf00 0x7f751814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 --2- 172.21.15.57:0/1886930722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f751806c800 0x7f75180aced0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 >> 172.21.15.57:0/1886930722 conn(0x7f751806b5d0 msgr2=0x7f7518159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 shutdown_connections 2023-10-07T23:21:26.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.574+0000 7f751c808700 1 -- 172.21.15.57:0/1886930722 wait complete. 2023-10-07T23:21:26.595 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T23:21:26.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 Processor -- start 2023-10-07T23:21:26.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 -- start start 2023-10-07T23:21:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fafa4159210 0x7fafa40ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa41595e0 0x7fafa406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa415a970 con 0x7fafa406cd40 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa406d1c0 con 0x7fafa4159210 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafab4b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40aced0 con 0x7fafa41595e0 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fafa4159210 0x7fafa40ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fafa4159210 0x7fafa40ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58154/0 (socket says 172.21.15.57:58154) 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 -- 172.21.15.57:0/1489628214 learned_addr learned my addr 172.21.15.57:0/1489628214 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa8a4e700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa41595e0 0x7fafa406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 -- 172.21.15.57:0/1489628214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa41595e0 msgr2=0x7fafa406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa41595e0 0x7fafa406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 -- 172.21.15.57:0/1489628214 >> v1:172.21.15.154:6789/0 conn(0x7fafa406cd40 legacy=0x7fafa40c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 -- 172.21.15.57:0/1489628214 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa40ad1b0 con 0x7fafa4159210 2023-10-07T23:21:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7fafa924f700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa40ad9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099b0 tx=0x7fafa0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e534c0313f9c5e68 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7faf9bfff700 1 -- 172.21.15.57:0/1489628214 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa000b860 con 0x7fafa4159210 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7faf9bfff700 1 -- 172.21.15.57:0/1489628214 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa000b9c0 con 0x7fafa4159210 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.706+0000 7faf9bfff700 1 -- 172.21.15.57:0/1489628214 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa000bd60 con 0x7fafa4159210 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 -- 172.21.15.57:0/1489628214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 msgr2=0x7fafa40ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa40ad9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099b0 tx=0x7fafa0009250 comp rx=0 tx=0).stop 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 -- 172.21.15.57:0/1489628214 shutdown_connections 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa41595e0 0x7fafa406c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 --2- 172.21.15.57:0/1489628214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa40ad9b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 -- 172.21.15.57:0/1489628214 >> 172.21.15.57:0/1489628214 conn(0x7fafa4062ba0 msgr2=0x7fafa4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 -- 172.21.15.57:0/1489628214 shutdown_connections 2023-10-07T23:21:26.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.710+0000 7fafab4b3700 1 -- 172.21.15.57:0/1489628214 wait complete. 2023-10-07T23:21:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 Processor -- start 2023-10-07T23:21:26.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 -- start start 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa406cd40 0x7fafa414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafa41595e0 0x7fafa40bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa4154cb0 con 0x7fafa4159210 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40af080 con 0x7fafa41595e0 2023-10-07T23:21:26.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafab4b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41549d0 con 0x7fafa406cd40 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58168/0 (socket says 172.21.15.57:58168) 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 -- 172.21.15.57:0/2745491580 learned_addr learned my addr 172.21.15.57:0/2745491580 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa924f700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa406cd40 0x7fafa414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa9a50700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafa41595e0 0x7fafa40bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 -- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa406cd40 msgr2=0x7fafa414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa406cd40 0x7fafa414d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 -- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafa41595e0 msgr2=0x7fafa40bf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafa41595e0 0x7fafa40bf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 -- 172.21.15.57:0/2745491580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa00094f0 con 0x7fafa4159210 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7fafa8a4e700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf9400d890 tx=0x7faf94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.714+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940125f0 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94012750 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa40aea30 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94012b10 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa41545c0 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa40647b0 con 0x7fafa4159210 2023-10-07T23:21:26.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faf9401e460 con 0x7fafa4159210 2023-10-07T23:21:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7faf997fa700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 0x7faf88062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.718+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faf9407ccf0 con 0x7fafa4159210 2023-10-07T23:21:26.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.722+0000 7fafa924f700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 0x7faf88062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.722+0000 7fafa924f700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 0x7faf88062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099b0 tx=0x7fafa0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:26.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.726+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faf94081070 con 0x7fafa4159210 2023-10-07T23:21:26.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.922+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fafa40af1c0 con 0x7fafa4159210 2023-10-07T23:21:26.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.938+0000 7faf997fa700 1 -- 172.21.15.57:0/2745491580 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7faf94050ae0 con 0x7fafa4159210 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.942+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 msgr2=0x7faf88062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.942+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 0x7faf88062a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafa00099b0 tx=0x7fafa0017000 comp rx=0 tx=0).stop 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.942+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 msgr2=0x7fafa4154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.942+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf9400d890 tx=0x7faf94008040 comp rx=0 tx=0).stop 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 shutdown_connections 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faf88060550 0x7faf88062a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fafa41595e0 0x7fafa40bf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fafa4159210 0x7fafa4154080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 --2- 172.21.15.57:0/2745491580 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fafa406cd40 0x7fafa414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:26.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 >> 172.21.15.57:0/2745491580 conn(0x7fafa4062ba0 msgr2=0x7fafa406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:26.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 shutdown_connections 2023-10-07T23:21:26.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:26.946+0000 7fafab4b3700 1 -- 172.21.15.57:0/2745491580 wait complete. 2023-10-07T23:21:26.965 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T23:21:27.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.074+0000 7fd7ec503700 1 Processor -- start 2023-10-07T23:21:27.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.074+0000 7fd7ec503700 1 -- start start 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 0x7fd7e40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd7e406c800 0x7fd7e406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 0x7fd7e40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e40b0450 con 0x7fd7e406c800 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e406a900 con 0x7fd7e40ca600 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e406a600 con 0x7fd7e4159030 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd7e406c800 0x7fd7e406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd7e406c800 0x7fd7e406cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58170/0 (socket says 172.21.15.57:58170) 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 -- 172.21.15.57:0/1378059654 learned_addr learned my addr 172.21.15.57:0/1378059654 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:27.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7eaaa0700 1 --2- 172.21.15.57:0/1378059654 >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 0x7fd7e40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 0x7fd7e40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 -- 172.21.15.57:0/1378059654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 msgr2=0x7fd7e40aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 0x7fd7e40aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 -- 172.21.15.57:0/1378059654 >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 msgr2=0x7fd7e40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 --2- 172.21.15.57:0/1378059654 >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 0x7fd7e40caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 -- 172.21.15.57:0/1378059654 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e406d170 con 0x7fd7e406c800 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7eaaa0700 1 --2- 172.21.15.57:0/1378059654 >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 0x7fd7e40caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 0x7fd7e40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e9a9e700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e406c800 0x7fd7e406cc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7e406a0f0 tx=0x7fd7e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15d43ac7ec4ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e929d700 1 -- 172.21.15.57:0/1378059654 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000b860 con 0x7fd7e406c800 2023-10-07T23:21:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e929d700 1 -- 172.21.15.57:0/1378059654 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000b9c0 con 0x7fd7e406c800 2023-10-07T23:21:27.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7e929d700 1 -- 172.21.15.57:0/1378059654 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000bd60 con 0x7fd7e406c800 2023-10-07T23:21:27.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- 172.21.15.57:0/1378059654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e406c800 msgr2=0x7fd7e406cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e406c800 0x7fd7e406cc30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7e406a0f0 tx=0x7fd7e0009250 comp rx=0 tx=0).stop 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- 172.21.15.57:0/1378059654 shutdown_connections 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- 172.21.15.57:0/1378059654 >> v2:172.21.15.154:3300/0 conn(0x7fd7e40ca600 0x7fd7e40caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e406c800 0x7fd7e406cc30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 --2- 172.21.15.57:0/1378059654 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e4159030 0x7fd7e40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- 172.21.15.57:0/1378059654 >> 172.21.15.57:0/1378059654 conn(0x7fd7e4068650 msgr2=0x7fd7e4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- 172.21.15.57:0/1378059654 shutdown_connections 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.078+0000 7fd7ec503700 1 -- 172.21.15.57:0/1378059654 wait complete. 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 Processor -- start 2023-10-07T23:21:27.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- start start 2023-10-07T23:21:27.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 0x7fd7e4152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 0x7fd7e4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e4152570 con 0x7fd7e4159030 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415b7f0 con 0x7fd7e40ca600 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40b4b00 con 0x7fd7e406c800 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58186/0 (socket says 172.21.15.57:58186) 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 -- 172.21.15.57:0/614548625 learned_addr learned my addr 172.21.15.57:0/614548625 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 0x7fd7e4152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7e9a9e700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 0x7fd7e4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 -- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 msgr2=0x7fd7e4152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 0x7fd7e4152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 -- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 msgr2=0x7fd7e4159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 0x7fd7e4159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 -- 172.21.15.57:0/614548625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00094f0 con 0x7fd7e4159030 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7e9a9e700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 0x7fd7e4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 0x7fd7e4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7eaaa0700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00cf90 tx=0x7fd7dc007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc015440 con 0x7fd7e4159030 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7dc0155a0 con 0x7fd7e4159030 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc015960 con 0x7fd7e4159030 2023-10-07T23:21:27.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e4159940 con 0x7fd7e4159030 2023-10-07T23:21:27.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.082+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e40c4720 con 0x7fd7e4159030 2023-10-07T23:21:27.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.086+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e40b4c90 con 0x7fd7e4159030 2023-10-07T23:21:27.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.086+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd7dc00d440 con 0x7fd7e4159030 2023-10-07T23:21:27.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.086+0000 7fd7d67fc700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 0x7fd7c8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:27.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.086+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7dc07dbe0 con 0x7fd7e4159030 2023-10-07T23:21:27.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.086+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 0x7fd7c8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:27.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.090+0000 7fd7ea29f700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 0x7fd7c8062a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:27.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.098+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7dc04e1d0 con 0x7fd7e4159030 2023-10-07T23:21:27.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.294+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd7e40c4940 con 0x7fd7e4159030 2023-10-07T23:21:27.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.962+0000 7fd7d67fc700 1 -- 172.21.15.57:0/614548625 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7dc01a0c0 con 0x7fd7e4159030 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 msgr2=0x7fd7c8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 0x7fd7c8062a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8006fd0 tx=0x7fd7d8008040 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 msgr2=0x7fd7e40c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00cf90 tx=0x7fd7dc007870 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 shutdown_connections 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd7e4159030 0x7fd7e40c3a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd7c8060580 0x7fd7c8062a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd7e40ca600 0x7fd7e4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 --2- 172.21.15.57:0/614548625 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd7e406c800 0x7fd7e4152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:27.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.966+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 >> 172.21.15.57:0/614548625 conn(0x7fd7e4068650 msgr2=0x7fd7e415cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:27.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.970+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 shutdown_connections 2023-10-07T23:21:27.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:27.970+0000 7fd7ec503700 1 -- 172.21.15.57:0/614548625 wait complete. 2023-10-07T23:21:27.990 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 Processor -- start 2023-10-07T23:21:28.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 -- start start 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f983c0aff20 0x7f983c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f983c15a0b0 0x7f983c0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15db90 con 0x7f983c0cc620 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c0cca40 con 0x7f983c0aff20 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f9841bcf700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983c15cf80 con 0x7f983c15a0b0 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f983c0aff20 0x7f983c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f983c0aff20 0x7f983c0b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58188/0 (socket says 172.21.15.57:58188) 2023-10-07T23:21:28.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 -- 172.21.15.57:0/1556870469 learned_addr learned my addr 172.21.15.57:0/1556870469 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:28.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983affd700 1 --2- 172.21.15.57:0/1556870469 >> v2:172.21.15.154:3300/0 conn(0x7f983c15a0b0 0x7f983c0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 -- 172.21.15.57:0/1556870469 >> v2:172.21.15.154:3300/0 conn(0x7f983c15a0b0 msgr2=0x7f983c0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 --2- 172.21.15.57:0/1556870469 >> v2:172.21.15.154:3300/0 conn(0x7f983c15a0b0 0x7f983c0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 -- 172.21.15.57:0/1556870469 >> v1:172.21.15.154:6789/0 conn(0x7f983c0cc620 legacy=0x7f983c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 -- 172.21.15.57:0/1556870469 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c15d260 con 0x7f983c0aff20 2023-10-07T23:21:28.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983b7fe700 1 --2- 172.21.15.57:0/1556870469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c0b02f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f983c0692c0 tx=0x7f982c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646a0b6f4a82ca54 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983a7fc700 1 -- 172.21.15.57:0/1556870469 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c009e70 con 0x7f983c0aff20 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983a7fc700 1 -- 172.21.15.57:0/1556870469 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c00b860 con 0x7f983c0aff20 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.102+0000 7f983a7fc700 1 -- 172.21.15.57:0/1556870469 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00bc00 con 0x7f983c0aff20 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- 172.21.15.57:0/1556870469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 msgr2=0x7f983c0b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1556870469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c0b02f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f983c0692c0 tx=0x7f982c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- 172.21.15.57:0/1556870469 shutdown_connections 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1556870469 >> v2:172.21.15.154:3300/0 conn(0x7f983c15a0b0 0x7f983c0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1556870469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c0b02f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- 172.21.15.57:0/1556870469 >> 172.21.15.57:0/1556870469 conn(0x7f983c0692f0 msgr2=0x7f983c0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- 172.21.15.57:0/1556870469 shutdown_connections 2023-10-07T23:21:28.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- 172.21.15.57:0/1556870469 wait complete. 2023-10-07T23:21:28.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 Processor -- start 2023-10-07T23:21:28.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.106+0000 7f9841bcf700 1 -- start start 2023-10-07T23:21:28.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f983c0cc620 0x7f983c0b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f983c15a0b0 0x7f983c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0aa0 con 0x7f983c0aff20 2023-10-07T23:21:28.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0d40 con 0x7f983c0cc620 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b1020 con 0x7f983c15a0b0 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58198/0 (socket says 172.21.15.57:58198) 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 -- 172.21.15.57:0/1216873506 learned_addr learned my addr 172.21.15.57:0/1216873506 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983affd700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f983c0cc620 0x7f983c0b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983bfff700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f983c15a0b0 0x7f983c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 -- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f983c15a0b0 msgr2=0x7f983c0b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f983c15a0b0 0x7f983c0b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 -- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f983c0cc620 msgr2=0x7f983c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f983c0cc620 0x7f983c0b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 -- 172.21.15.57:0/1216873506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0094f0 con 0x7f983c0aff20 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f983b7fe700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f983c06b300 tx=0x7f982c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00bbe0 con 0x7f983c0aff20 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c000c90 con 0x7f983c0aff20 2023-10-07T23:21:28.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c0b1300 con 0x7f983c0aff20 2023-10-07T23:21:28.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c011670 con 0x7f983c0aff20 2023-10-07T23:21:28.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c0b18c0 con 0x7f983c0aff20 2023-10-07T23:21:28.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.110+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c0711c0 con 0x7f983c0aff20 2023-10-07T23:21:28.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.114+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f982c011a60 con 0x7f983c0aff20 2023-10-07T23:21:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.114+0000 7f98337fe700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 0x7f981c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.114+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f982c07dd60 con 0x7f983c0aff20 2023-10-07T23:21:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.114+0000 7f983affd700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 0x7f981c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.114+0000 7f983affd700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 0x7f981c062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f982400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.118+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f982c04e2b0 con 0x7f983c0aff20 2023-10-07T23:21:28.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.314+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f983c15d540 con 0x7f983c0aff20 2023-10-07T23:21:28.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.314+0000 7f98337fe700 1 -- 172.21.15.57:0/1216873506 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f982c0518d0 con 0x7f983c0aff20 2023-10-07T23:21:28.321 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:28.321 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 msgr2=0x7f981c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 0x7f981c062a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f982400b040 comp rx=0 tx=0).stop 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 msgr2=0x7f983c158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f983c06b300 tx=0x7f982c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 shutdown_connections 2023-10-07T23:21:28.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f983c15a0b0 0x7f983c0b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f981c060550 0x7f981c062a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f983c0cc620 0x7f983c0b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 --2- 172.21.15.57:0/1216873506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f983c0aff20 0x7f983c158d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 >> 172.21.15.57:0/1216873506 conn(0x7f983c0692f0 msgr2=0x7f983c0c2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 shutdown_connections 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.318+0000 7f9841bcf700 1 -- 172.21.15.57:0/1216873506 wait complete. 2023-10-07T23:21:28.324 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:28.335 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 Processor -- start 2023-10-07T23:21:28.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 -- start start 2023-10-07T23:21:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 0x7f10d4159290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f10d40b0d40 0x7f10d40a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d40ae0a0 con 0x7f10d40ac710 2023-10-07T23:21:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d40ad250 con 0x7f10d40b0d40 2023-10-07T23:21:28.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40ad490 con 0x7f10d4158ec0 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d1d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f10d40ac710 0x7f10d40acb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:49772/0 (socket says 172.21.15.57:49772) 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d1d9b700 1 -- 172.21.15.57:0/1678484616 learned_addr learned my addr 172.21.15.57:0/1678484616 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 --2- 172.21.15.57:0/1678484616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 0x7f10d4159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d2d9d700 1 --2- 172.21.15.57:0/1678484616 >> v2:172.21.15.154:3300/0 conn(0x7f10d40b0d40 0x7f10d40a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d159a700 1 -- 172.21.15.57:0/1678484616 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887777414 0 0) 0x7f10d40ae0a0 con 0x7f10d40ac710 2023-10-07T23:21:28.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d159a700 1 -- 172.21.15.57:0/1678484616 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c0003570 con 0x7f10d40ac710 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 -- 172.21.15.57:0/1678484616 >> v2:172.21.15.154:3300/0 conn(0x7f10d40b0d40 msgr2=0x7f10d40a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 --2- 172.21.15.57:0/1678484616 >> v2:172.21.15.154:3300/0 conn(0x7f10d40b0d40 0x7f10d40a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 -- 172.21.15.57:0/1678484616 >> v1:172.21.15.154:6789/0 conn(0x7f10d40ac710 legacy=0x7f10d40acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 -- 172.21.15.57:0/1678484616 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40ad770 con 0x7f10d4158ec0 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d259c700 1 --2- 172.21.15.57:0/1678484616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 0x7f10d4159290 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0098e0 tx=0x7f10bc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f248b2973c73a90 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d159a700 1 -- 172.21.15.57:0/1678484616 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc011630 con 0x7f10d4158ec0 2023-10-07T23:21:28.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d159a700 1 -- 172.21.15.57:0/1678484616 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc011790 con 0x7f10d4158ec0 2023-10-07T23:21:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.438+0000 7f10d159a700 1 -- 172.21.15.57:0/1678484616 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc011b10 con 0x7f10d4158ec0 2023-10-07T23:21:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- 172.21.15.57:0/1678484616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 msgr2=0x7f10d4159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- 172.21.15.57:0/1678484616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 0x7f10d4159290 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0098e0 tx=0x7f10bc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- 172.21.15.57:0/1678484616 shutdown_connections 2023-10-07T23:21:28.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- 172.21.15.57:0/1678484616 >> v2:172.21.15.154:3300/0 conn(0x7f10d40b0d40 0x7f10d40a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- 172.21.15.57:0/1678484616 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d4158ec0 0x7f10d4159290 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- 172.21.15.57:0/1678484616 >> 172.21.15.57:0/1678484616 conn(0x7f10d406e0f0 msgr2=0x7f10d4095a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- 172.21.15.57:0/1678484616 shutdown_connections 2023-10-07T23:21:28.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- 172.21.15.57:0/1678484616 wait complete. 2023-10-07T23:21:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 Processor -- start 2023-10-07T23:21:28.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- start start 2023-10-07T23:21:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d40b0d40 0x7f10d4144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d4158ec0 0x7f10d4145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4145990 con 0x7f10d40ac710 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d414ab80 con 0x7f10d4158ec0 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d4149cd0 con 0x7f10d40b0d40 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58214/0 (socket says 172.21.15.57:58214) 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d259c700 1 -- 172.21.15.57:0/137757526 learned_addr learned my addr 172.21.15.57:0/137757526 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d1d9b700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d40b0d40 0x7f10d4144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.442+0000 7f10d2d9d700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d4158ec0 0x7f10d4145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 -- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d40b0d40 msgr2=0x7f10d4144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d40b0d40 0x7f10d4144f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 -- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d4158ec0 msgr2=0x7f10d4145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d4158ec0 0x7f10d4145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 -- 172.21.15.57:0/137757526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc009420 con 0x7f10d40ac710 2023-10-07T23:21:28.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d259c700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ad5d0 tx=0x7f10bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc011900 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10bc011a60 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d4149f70 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc01a680 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d414a5b0 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d40711c0 con 0x7f10d40ac710 2023-10-07T23:21:28.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.446+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f10bc01aa30 con 0x7f10d40ac710 2023-10-07T23:21:28.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.450+0000 7f10ca7fc700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 0x7f10b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.450+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f10bc07e3c0 con 0x7f10d40ac710 2023-10-07T23:21:28.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.450+0000 7f10d1d9b700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 0x7f10b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.450+0000 7f10d1d9b700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 0x7f10b4062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10d416a1c0 tx=0x7f10cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.458+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f10bc04e9b0 con 0x7f10d40ac710 2023-10-07T23:21:28.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.622+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10d40ada50 con 0x7f10d40ac710 2023-10-07T23:21:28.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.622+0000 7f10ca7fc700 1 -- 172.21.15.57:0/137757526 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f10bc051fd0 con 0x7f10d40ac710 2023-10-07T23:21:28.629 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:28.629 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 msgr2=0x7f10b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 0x7f10b4062a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f10d416a1c0 tx=0x7f10cc009040 comp rx=0 tx=0).stop 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 msgr2=0x7f10d4147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ad5d0 tx=0x7f10bc017000 comp rx=0 tx=0).stop 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 shutdown_connections 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f10b4060550 0x7f10b4062a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10d4158ec0 0x7f10d4145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f10d40b0d40 0x7f10d4144f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 --2- 172.21.15.57:0/137757526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f10d40ac710 0x7f10d4147d10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 >> 172.21.15.57:0/137757526 conn(0x7f10d406e0f0 msgr2=0x7f10d41506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 shutdown_connections 2023-10-07T23:21:28.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.626+0000 7f10d359e700 1 -- 172.21.15.57:0/137757526 wait complete. 2023-10-07T23:21:28.643 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T23:21:28.643 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:21:28.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 Processor -- start 2023-10-07T23:21:28.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 -- start start 2023-10-07T23:21:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f4406bc90 0x7f2f440c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f440c83b0 con 0x7f2f44158d60 2023-10-07T23:21:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440c8690 con 0x7f2f4406bc90 2023-10-07T23:21:28.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f49fad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440c8970 con 0x7f2f44159130 2023-10-07T23:21:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f4406bc90 0x7f2f440c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f497ac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f48fab700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2f44158d60 0x7f2f4406b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:49778/0 (socket says 172.21.15.57:49778) 2023-10-07T23:21:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f48fab700 1 -- 172.21.15.57:0/1533247790 learned_addr learned my addr 172.21.15.57:0/1533247790 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:28.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1533247790 >> v2:172.21.15.154:3300/0 conn(0x7f2f4406bc90 0x7f2f440c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36796/0 (socket says 172.21.15.57:36796) 2023-10-07T23:21:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 -- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 msgr2=0x7f2f440c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 -- 172.21.15.57:0/1533247790 >> v1:172.21.15.154:6789/0 conn(0x7f2f44158d60 legacy=0x7f2f4406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f437fe700 1 -- 172.21.15.57:0/1533247790 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008029279 0 0) 0x7f2f440c83b0 con 0x7f2f44158d60 2023-10-07T23:21:28.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.746+0000 7f2f43fff700 1 -- 172.21.15.57:0/1533247790 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440c8c50 con 0x7f2f4406bc90 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406bc90 0x7f2f440c7580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bb90 tx=0x7f2f2c00b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c07068cb26218190 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f437fe700 1 -- 172.21.15.57:0/1533247790 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00db40 con 0x7f2f4406bc90 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f437fe700 1 -- 172.21.15.57:0/1533247790 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c00dca0 con 0x7f2f4406bc90 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f437fe700 1 -- 172.21.15.57:0/1533247790 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0196b0 con 0x7f2f4406bc90 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406bc90 msgr2=0x7f2f440c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406bc90 0x7f2f440c7580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bb90 tx=0x7f2f2c00b430 comp rx=0 tx=0).stop 2023-10-07T23:21:28.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- 172.21.15.57:0/1533247790 shutdown_connections 2023-10-07T23:21:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1533247790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4406bc90 0x7f2f440c7580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- 172.21.15.57:0/1533247790 >> 172.21.15.57:0/1533247790 conn(0x7f2f440670e0 msgr2=0x7f2f440674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- 172.21.15.57:0/1533247790 shutdown_connections 2023-10-07T23:21:28.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- 172.21.15.57:0/1533247790 wait complete. 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 Processor -- start 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- start start 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f44158d60 0x7f2f441544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440ae860 con 0x7f2f4406bc90 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440af3e0 con 0x7f2f44158d60 2023-10-07T23:21:28.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f49fad700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440aeab0 con 0x7f2f44159130 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f48fab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f48fab700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58220/0 (socket says 172.21.15.57:58220) 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f48fab700 1 -- 172.21.15.57:0/1839452899 learned_addr learned my addr 172.21.15.57:0/1839452899 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f44158d60 0x7f2f441544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.750+0000 7f2f497ac700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 -- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 msgr2=0x7f2f440cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 -- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f44158d60 msgr2=0x7f2f441544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f44158d60 0x7f2f441544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 -- 172.21.15.57:0/1839452899 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c00b6d0 con 0x7f2f4406bc90 2023-10-07T23:21:28.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f48fab700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400cdb0 tx=0x7f2f34001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400ab10 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400ac70 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f440aed90 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340070f0 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44154a20 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f440711c0 con 0x7f2f4406bc90 2023-10-07T23:21:28.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2f340074e0 con 0x7f2f4406bc90 2023-10-07T23:21:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.754+0000 7f2f40ff9700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 0x7f2f24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:28.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.758+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f3407cd30 con 0x7f2f4406bc90 2023-10-07T23:21:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.758+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 0x7f2f24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:28.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.758+0000 7f2f43fff700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 0x7f2f24062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406c090 tx=0x7f2f2c01b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:28.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.762+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f3404d280 con 0x7f2f4406bc90 2023-10-07T23:21:28.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.922+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f440af520 con 0x7f2f24060550 2023-10-07T23:21:28.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.942+0000 7f2f40ff9700 1 -- 172.21.15.57:0/1839452899 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251523 (secure 0 0 0) 0x7f2f2c01e040 con 0x7f2f24060550 2023-10-07T23:21:28.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 msgr2=0x7f2f24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 0x7f2f24062a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f4406c090 tx=0x7f2f2c01b000 comp rx=0 tx=0).stop 2023-10-07T23:21:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 msgr2=0x7f2f44149cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400cdb0 tx=0x7f2f34001ac0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 shutdown_connections 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f44159130 0x7f2f440cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2f24060550 0x7f2f24062a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f44158d60 0x7f2f441544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 --2- 172.21.15.57:0/1839452899 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f4406bc90 0x7f2f44149cc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 >> 172.21.15.57:0/1839452899 conn(0x7f2f440670e0 msgr2=0x7f2f44154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 shutdown_connections 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:28.946+0000 7f2f49fad700 1 -- 172.21.15.57:0/1839452899 wait complete. 2023-10-07T23:21:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:21:28.965 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-07T23:21:27.070247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":164,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":2397460,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577260,"statfs":{"total":773094113280,"available":770639114240,"internally_reserved":0,"allocated":2707456,"data_stored":1643823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001290"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.817564+0000","last_change":"2023-10-07T23:21:25.817564+0000","last_active":"2023-10-07T23:21:25.817564+0000","last_peered":"2023-10-07T23:21:25.817564+0000","last_clean":"2023-10-07T23:21:25.817564+0000","last_became_active":"2023-10-07T23:21:25.817149+0000","last_became_peered":"2023-10-07T23:21:25.817149+0000","last_unstale":"2023-10-07T23:21:25.817564+0000","last_undegraded":"2023-10-07T23:21:25.817564+0000","last_fullsized":"2023-10-07T23:21:25.817564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:32.392634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815145+0000","last_change":"2023-10-07T23:21:25.815145+0000","last_active":"2023-10-07T23:21:25.815145+0000","last_peered":"2023-10-07T23:21:25.815145+0000","last_clean":"2023-10-07T23:21:25.815145+0000","last_became_active":"2023-10-07T23:21:25.814843+0000","last_became_peered":"2023-10-07T23:21:25.814843+0000","last_unstale":"2023-10-07T23:21:25.815145+0000","last_undegraded":"2023-10-07T23:21:25.815145+0000","last_fullsized":"2023-10-07T23:21:25.815145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:37.712425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816989+0000","last_change":"2023-10-07T23:21:25.816989+0000","last_active":"2023-10-07T23:21:25.816989+0000","last_peered":"2023-10-07T23:21:25.816989+0000","last_clean":"2023-10-07T23:21:25.816989+0000","last_became_active":"2023-10-07T23:21:25.816637+0000","last_became_peered":"2023-10-07T23:21:25.816637+0000","last_unstale":"2023-10-07T23:21:25.816989+0000","last_undegraded":"2023-10-07T23:21:25.816989+0000","last_fullsized":"2023-10-07T23:21:25.816989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:34.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816941+0000","last_change":"2023-10-07T23:21:25.816941+0000","last_active":"2023-10-07T23:21:25.816941+0000","last_peered":"2023-10-07T23:21:25.816941+0000","last_clean":"2023-10-07T23:21:25.816941+0000","last_became_active":"2023-10-07T23:21:25.816658+0000","last_became_peered":"2023-10-07T23:21:25.816658+0000","last_unstale":"2023-10-07T23:21:25.816941+0000","last_undegraded":"2023-10-07T23:21:25.816941+0000","last_fullsized":"2023-10-07T23:21:25.816941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:20.455188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954772+0000","last_change":"2023-10-07T23:21:24.954956+0000","last_active":"2023-10-07T23:21:25.954772+0000","last_peered":"2023-10-07T23:21:25.954772+0000","last_clean":"2023-10-07T23:21:25.954772+0000","last_became_active":"2023-10-07T23:21:24.954596+0000","last_became_peered":"2023-10-07T23:21:24.954596+0000","last_unstale":"2023-10-07T23:21:25.954772+0000","last_undegraded":"2023-10-07T23:21:25.954772+0000","last_fullsized":"2023-10-07T23:21:25.954772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:02.562726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.800876+0000","last_change":"2023-10-07T23:21:24.948082+0000","last_active":"2023-10-07T23:21:25.800876+0000","last_peered":"2023-10-07T23:21:25.800876+0000","last_clean":"2023-10-07T23:21:25.800876+0000","last_became_active":"2023-10-07T23:21:24.947759+0000","last_became_peered":"2023-10-07T23:21:24.947759+0000","last_unstale":"2023-10-07T23:21:25.800876+0000","last_undegraded":"2023-10-07T23:21:25.800876+0000","last_fullsized":"2023-10-07T23:21:25.800876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:11.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":159,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-07T23:21:16.736832+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:21:16.736832+0000","last_peered":"2023-10-07T23:21:16.736832+0000","last_clean":"2023-10-07T23:21:16.736832+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:21:16.736832+0000","last_undegraded":"2023-10-07T23:21:16.736832+0000","last_fullsized":"2023-10-07T23:21:16.736832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815004+0000","last_change":"2023-10-07T23:21:25.815004+0000","last_active":"2023-10-07T23:21:25.815004+0000","last_peered":"2023-10-07T23:21:25.815004+0000","last_clean":"2023-10-07T23:21:25.815004+0000","last_became_active":"2023-10-07T23:21:25.814688+0000","last_became_peered":"2023-10-07T23:21:25.814688+0000","last_unstale":"2023-10-07T23:21:25.815004+0000","last_undegraded":"2023-10-07T23:21:25.815004+0000","last_fullsized":"2023-10-07T23:21:25.815004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:46.190235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.960593+0000","last_change":"2023-10-07T23:21:24.952562+0000","last_active":"2023-10-07T23:21:24.960593+0000","last_peered":"2023-10-07T23:21:24.960593+0000","last_clean":"2023-10-07T23:21:24.960593+0000","last_became_active":"2023-10-07T23:21:24.952071+0000","last_became_peered":"2023-10-07T23:21:24.952071+0000","last_unstale":"2023-10-07T23:21:24.960593+0000","last_undegraded":"2023-10-07T23:21:24.960593+0000","last_fullsized":"2023-10-07T23:21:24.960593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:17.603304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815292+0000","last_change":"2023-10-07T23:21:25.815292+0000","last_active":"2023-10-07T23:21:25.815292+0000","last_peered":"2023-10-07T23:21:25.815292+0000","last_clean":"2023-10-07T23:21:25.815292+0000","last_became_active":"2023-10-07T23:21:25.814294+0000","last_became_peered":"2023-10-07T23:21:25.814294+0000","last_unstale":"2023-10-07T23:21:25.815292+0000","last_undegraded":"2023-10-07T23:21:25.815292+0000","last_fullsized":"2023-10-07T23:21:25.815292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:41.638201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.952777+0000","last_change":"2023-10-07T23:21:24.948131+0000","last_active":"2023-10-07T23:21:24.952777+0000","last_peered":"2023-10-07T23:21:24.952777+0000","last_clean":"2023-10-07T23:21:24.952777+0000","last_became_active":"2023-10-07T23:21:24.947831+0000","last_became_peered":"2023-10-07T23:21:24.947831+0000","last_unstale":"2023-10-07T23:21:24.952777+0000","last_undegraded":"2023-10-07T23:21:24.952777+0000","last_fullsized":"2023-10-07T23:21:24.952777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:23.995859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.961205+0000","last_change":"2023-10-07T23:21:24.958141+0000","last_active":"2023-10-07T23:21:24.961205+0000","last_peered":"2023-10-07T23:21:24.961205+0000","last_clean":"2023-10-07T23:21:24.961205+0000","last_became_active":"2023-10-07T23:21:24.957791+0000","last_became_peered":"2023-10-07T23:21:24.957791+0000","last_unstale":"2023-10-07T23:21:24.961205+0000","last_undegraded":"2023-10-07T23:21:24.961205+0000","last_fullsized":"2023-10-07T23:21:24.961205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:01.467025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.961237+0000","last_change":"2023-10-07T23:21:24.957341+0000","last_active":"2023-10-07T23:21:24.961237+0000","last_peered":"2023-10-07T23:21:24.961237+0000","last_clean":"2023-10-07T23:21:24.961237+0000","last_became_active":"2023-10-07T23:21:24.957059+0000","last_became_peered":"2023-10-07T23:21:24.957059+0000","last_unstale":"2023-10-07T23:21:24.961237+0000","last_undegraded":"2023-10-07T23:21:24.961237+0000","last_fullsized":"2023-10-07T23:21:24.961237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.702939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:24.912407+0000","last_change":"2023-10-07T23:21:24.912407+0000","last_active":"2023-10-07T23:21:24.912407+0000","last_peered":"2023-10-07T23:21:24.912407+0000","last_clean":"2023-10-07T23:21:24.912407+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:24.912407+0000","last_undegraded":"2023-10-07T23:21:24.912407+0000","last_fullsized":"2023-10-07T23:21:24.912407+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:21:23.899009+0000","last_change":"2023-10-07T23:21:23.899009+0000","last_active":"2023-10-07T23:21:23.899009+0000","last_peered":"2023-10-07T23:21:23.899009+0000","last_clean":"2023-10-07T23:21:23.899009+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:21:23.899009+0000","last_undegraded":"2023-10-07T23:21:23.899009+0000","last_fullsized":"2023-10-07T23:21:23.899009+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":12,"acting":12,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":645880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297468,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074372,"statfs":{"total":96636764160,"available":96332156928,"internally_reserved":0,"allocated":184320,"data_stored":55512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297468,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074372,"statfs":{"total":96636764160,"available":96332156928,"internally_reserved":0,"allocated":184320,"data_stored":55512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]}]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297468,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074372,"statfs":{"total":96636764160,"available":96332156928,"internally_reserved":0,"allocated":184320,"data_stored":55512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301564,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070276,"statfs":{"total":96636764160,"available":96327962624,"internally_reserved":0,"allocated":184320,"data_stored":55512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301564,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070276,"statfs":{"total":96636764160,"available":96327962624,"internally_reserved":0,"allocated":184320,"data_stored":55512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302272,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069568,"statfs":{"total":96636764160,"available":96327237632,"internally_reserved":0,"allocated":778240,"data_stored":645880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.359}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":74503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:21:28.971 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:21:29.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 Processor -- start 2023-10-07T23:21:29.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- start start 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8fb4158fb0 0x7f8fb40d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb406b3c0 con 0x7f8fb4158be0 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb406a7b0 con 0x7f8fb40ce880 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb406aa90 con 0x7f8fb4158fb0 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8fb4158fb0 0x7f8fb40d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb99dc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8fb4158be0 0x7f8fb40ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60432/0 (socket says 172.21.15.57:60432) 2023-10-07T23:21:29.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8fb4158fb0 0x7f8fb40d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58224/0 (socket says 172.21.15.57:58224) 2023-10-07T23:21:29.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb99dc700 1 -- 172.21.15.57:0/2588133643 learned_addr learned my addr 172.21.15.57:0/2588133643 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:29.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 -- 172.21.15.57:0/2588133643 >> v1:172.21.15.154:6789/0 conn(0x7f8fb40ce880 legacy=0x7f8fb40cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb89da700 1 -- 172.21.15.57:0/2588133643 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502019530 0 0) 0x7f8fb406b3c0 con 0x7f8fb4158be0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 -- 172.21.15.57:0/2588133643 >> v1:172.21.15.57:6789/0 conn(0x7f8fb4158be0 legacy=0x7f8fb40ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 -- 172.21.15.57:0/2588133643 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb406ad70 con 0x7f8fb4158fb0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba1dd700 1 --2- 172.21.15.57:0/2588133643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb40d0330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009e90 tx=0x7f8fac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcab1c76992e99e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb89da700 1 -- 172.21.15.57:0/2588133643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00e070 con 0x7f8fb4158fb0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb89da700 1 -- 172.21.15.57:0/2588133643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac00bbf0 con 0x7f8fb4158fb0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fb89da700 1 -- 172.21.15.57:0/2588133643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac012430 con 0x7f8fb4158fb0 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- 172.21.15.57:0/2588133643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 msgr2=0x7f8fb40d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 --2- 172.21.15.57:0/2588133643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb40d0330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009e90 tx=0x7f8fac009780 comp rx=0 tx=0).stop 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- 172.21.15.57:0/2588133643 shutdown_connections 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 --2- 172.21.15.57:0/2588133643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb40d0330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.074+0000 7f8fba9de700 1 -- 172.21.15.57:0/2588133643 >> 172.21.15.57:0/2588133643 conn(0x7f8fb4068520 msgr2=0x7f8fb4068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- 172.21.15.57:0/2588133643 shutdown_connections 2023-10-07T23:21:29.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- 172.21.15.57:0/2588133643 wait complete. 2023-10-07T23:21:29.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 Processor -- start 2023-10-07T23:21:29.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- start start 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40ce880 0x7f8fb40c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb4158be0 0x7f8fb41468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4144030 con 0x7f8fb4158fb0 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40c7720 con 0x7f8fb4158be0 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.078+0000 7f8fba9de700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40c7160 con 0x7f8fb40ce880 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58240/0 (socket says 172.21.15.57:58240) 2023-10-07T23:21:29.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 -- 172.21.15.57:0/3665606517 learned_addr learned my addr 172.21.15.57:0/3665606517 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fb99dc700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40ce880 0x7f8fb40c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fb91db700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb4158be0 0x7f8fb41468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 -- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40ce880 msgr2=0x7f8fb40c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40ce880 0x7f8fb40c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 -- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb4158be0 msgr2=0x7f8fb41468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb4158be0 0x7f8fb41468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 -- 172.21.15.57:0/3665606517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009a20 con 0x7f8fb4158fb0 2023-10-07T23:21:29.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba1dd700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406ec20 tx=0x7f8fac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01b630 con 0x7f8fb4158fb0 2023-10-07T23:21:29.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac01b790 con 0x7f8fb4158fb0 2023-10-07T23:21:29.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb40c7440 con 0x7f8fb4158fb0 2023-10-07T23:21:29.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac01bb50 con 0x7f8fb4158fb0 2023-10-07T23:21:29.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4156310 con 0x7f8fb4158fb0 2023-10-07T23:21:29.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb4156530 con 0x7f8fb4158fb0 2023-10-07T23:21:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fac01c400 con 0x7f8fb4158fb0 2023-10-07T23:21:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.082+0000 7f8fa5ffb700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 0x7f8f98062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.086+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fac07e710 con 0x7f8fb4158fb0 2023-10-07T23:21:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.086+0000 7f8fb99dc700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 0x7f8f98062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.086+0000 7f8fb99dc700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 0x7f8f98062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00011e0 tx=0x7f8fb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.094+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fac04ed00 con 0x7f8fb4158fb0 2023-10-07T23:21:29.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.258+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fb40711c0 con 0x7f8f98060550 2023-10-07T23:21:29.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.270+0000 7f8fa5ffb700 1 -- 172.21.15.57:0/3665606517 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264449 (secure 0 0 0) 0x7f8fb40711c0 con 0x7f8f98060550 2023-10-07T23:21:29.280 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.274+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 msgr2=0x7f8f98062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.274+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 0x7f8f98062a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00011e0 tx=0x7f8fb0009040 comp rx=0 tx=0).stop 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.274+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 msgr2=0x7f8fb416c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.274+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406ec20 tx=0x7f8fac019000 comp rx=0 tx=0).stop 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 shutdown_connections 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fb4158fb0 0x7f8fb416c8a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f98060550 0x7f8f98062a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fb4158be0 0x7f8fb41468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 --2- 172.21.15.57:0/3665606517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fb40ce880 0x7f8fb40c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 >> 172.21.15.57:0/3665606517 conn(0x7f8fb4068520 msgr2=0x7f8fb4154840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 shutdown_connections 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.278+0000 7f8fba9de700 1 -- 172.21.15.57:0/3665606517 wait complete. 2023-10-07T23:21:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:21:29.295 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-07T23:21:29.070838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397640,"kb_used_data":2952,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577080,"statfs":{"total":773094113280,"available":770638929920,"internally_reserved":0,"allocated":3022848,"data_stored":1780092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.154223"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824034+0000","last_change":"2023-10-07T23:21:25.817169+0000","last_active":"2023-10-07T23:21:25.824034+0000","last_peered":"2023-10-07T23:21:25.824034+0000","last_clean":"2023-10-07T23:21:25.824034+0000","last_became_active":"2023-10-07T23:21:25.816152+0000","last_became_peered":"2023-10-07T23:21:25.816152+0000","last_unstale":"2023-10-07T23:21:25.824034+0000","last_undegraded":"2023-10-07T23:21:25.824034+0000","last_fullsized":"2023-10-07T23:21:25.824034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.817564+0000","last_change":"2023-10-07T23:21:25.817564+0000","last_active":"2023-10-07T23:21:25.817564+0000","last_peered":"2023-10-07T23:21:25.817564+0000","last_clean":"2023-10-07T23:21:25.817564+0000","last_became_active":"2023-10-07T23:21:25.817149+0000","last_became_peered":"2023-10-07T23:21:25.817149+0000","last_unstale":"2023-10-07T23:21:25.817564+0000","last_undegraded":"2023-10-07T23:21:25.817564+0000","last_fullsized":"2023-10-07T23:21:25.817564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:32.392634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823467+0000","last_change":"2023-10-07T23:21:25.813864+0000","last_active":"2023-10-07T23:21:25.823467+0000","last_peered":"2023-10-07T23:21:25.823467+0000","last_clean":"2023-10-07T23:21:25.823467+0000","last_became_active":"2023-10-07T23:21:25.813627+0000","last_became_peered":"2023-10-07T23:21:25.813627+0000","last_unstale":"2023-10-07T23:21:25.823467+0000","last_undegraded":"2023-10-07T23:21:25.823467+0000","last_fullsized":"2023-10-07T23:21:25.823467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822927+0000","last_change":"2023-10-07T23:21:25.817746+0000","last_active":"2023-10-07T23:21:25.822927+0000","last_peered":"2023-10-07T23:21:25.822927+0000","last_clean":"2023-10-07T23:21:25.822927+0000","last_became_active":"2023-10-07T23:21:25.817419+0000","last_became_peered":"2023-10-07T23:21:25.817419+0000","last_unstale":"2023-10-07T23:21:25.822927+0000","last_undegraded":"2023-10-07T23:21:25.822927+0000","last_fullsized":"2023-10-07T23:21:25.822927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824136+0000","last_change":"2023-10-07T23:21:25.817774+0000","last_active":"2023-10-07T23:21:25.824136+0000","last_peered":"2023-10-07T23:21:25.824136+0000","last_clean":"2023-10-07T23:21:25.824136+0000","last_became_active":"2023-10-07T23:21:25.817279+0000","last_became_peered":"2023-10-07T23:21:25.817279+0000","last_unstale":"2023-10-07T23:21:25.824136+0000","last_undegraded":"2023-10-07T23:21:25.824136+0000","last_fullsized":"2023-10-07T23:21:25.824136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824103+0000","last_change":"2023-10-07T23:21:25.821044+0000","last_active":"2023-10-07T23:21:25.824103+0000","last_peered":"2023-10-07T23:21:25.824103+0000","last_clean":"2023-10-07T23:21:25.824103+0000","last_became_active":"2023-10-07T23:21:25.820744+0000","last_became_peered":"2023-10-07T23:21:25.820744+0000","last_unstale":"2023-10-07T23:21:25.824103+0000","last_undegraded":"2023-10-07T23:21:25.824103+0000","last_fullsized":"2023-10-07T23:21:25.824103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.815381+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.815050+0000","last_became_peered":"2023-10-07T23:21:25.815050+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824873+0000","last_change":"2023-10-07T23:21:25.817306+0000","last_active":"2023-10-07T23:21:25.824873+0000","last_peered":"2023-10-07T23:21:25.824873+0000","last_clean":"2023-10-07T23:21:25.824873+0000","last_became_active":"2023-10-07T23:21:25.816989+0000","last_became_peered":"2023-10-07T23:21:25.816989+0000","last_unstale":"2023-10-07T23:21:25.824873+0000","last_undegraded":"2023-10-07T23:21:25.824873+0000","last_fullsized":"2023-10-07T23:21:25.824873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823714+0000","last_change":"2023-10-07T23:21:25.815302+0000","last_active":"2023-10-07T23:21:25.823714+0000","last_peered":"2023-10-07T23:21:25.823714+0000","last_clean":"2023-10-07T23:21:25.823714+0000","last_became_active":"2023-10-07T23:21:25.814105+0000","last_became_peered":"2023-10-07T23:21:25.814105+0000","last_unstale":"2023-10-07T23:21:25.823714+0000","last_undegraded":"2023-10-07T23:21:25.823714+0000","last_fullsized":"2023-10-07T23:21:25.823714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:37.211418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.805674+0000","last_change":"2023-10-07T23:21:25.800296+0000","last_active":"2023-10-07T23:21:25.805674+0000","last_peered":"2023-10-07T23:21:25.805674+0000","last_clean":"2023-10-07T23:21:25.805674+0000","last_became_active":"2023-10-07T23:21:25.799882+0000","last_became_peered":"2023-10-07T23:21:25.799882+0000","last_unstale":"2023-10-07T23:21:25.805674+0000","last_undegraded":"2023-10-07T23:21:25.805674+0000","last_fullsized":"2023-10-07T23:21:25.805674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806603+0000","last_change":"2023-10-07T23:21:25.803198+0000","last_active":"2023-10-07T23:21:25.806603+0000","last_peered":"2023-10-07T23:21:25.806603+0000","last_clean":"2023-10-07T23:21:25.806603+0000","last_became_active":"2023-10-07T23:21:25.802597+0000","last_became_peered":"2023-10-07T23:21:25.802597+0000","last_unstale":"2023-10-07T23:21:25.806603+0000","last_undegraded":"2023-10-07T23:21:25.806603+0000","last_fullsized":"2023-10-07T23:21:25.806603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816488+0000","last_change":"2023-10-07T23:21:25.809325+0000","last_active":"2023-10-07T23:21:25.816488+0000","last_peered":"2023-10-07T23:21:25.816488+0000","last_clean":"2023-10-07T23:21:25.816488+0000","last_became_active":"2023-10-07T23:21:25.808944+0000","last_became_peered":"2023-10-07T23:21:25.808944+0000","last_unstale":"2023-10-07T23:21:25.816488+0000","last_undegraded":"2023-10-07T23:21:25.816488+0000","last_fullsized":"2023-10-07T23:21:25.816488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:50.633181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822948+0000","last_change":"2023-10-07T23:21:25.815656+0000","last_active":"2023-10-07T23:21:25.822948+0000","last_peered":"2023-10-07T23:21:25.822948+0000","last_clean":"2023-10-07T23:21:25.822948+0000","last_became_active":"2023-10-07T23:21:25.815290+0000","last_became_peered":"2023-10-07T23:21:25.815290+0000","last_unstale":"2023-10-07T23:21:25.822948+0000","last_undegraded":"2023-10-07T23:21:25.822948+0000","last_fullsized":"2023-10-07T23:21:25.822948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831315+0000","last_change":"2023-10-07T23:21:25.821314+0000","last_active":"2023-10-07T23:21:25.831315+0000","last_peered":"2023-10-07T23:21:25.831315+0000","last_clean":"2023-10-07T23:21:25.831315+0000","last_became_active":"2023-10-07T23:21:25.820716+0000","last_became_peered":"2023-10-07T23:21:25.820716+0000","last_unstale":"2023-10-07T23:21:25.831315+0000","last_undegraded":"2023-10-07T23:21:25.831315+0000","last_fullsized":"2023-10-07T23:21:25.831315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815145+0000","last_change":"2023-10-07T23:21:25.815145+0000","last_active":"2023-10-07T23:21:25.815145+0000","last_peered":"2023-10-07T23:21:25.815145+0000","last_clean":"2023-10-07T23:21:25.815145+0000","last_became_active":"2023-10-07T23:21:25.814843+0000","last_became_peered":"2023-10-07T23:21:25.814843+0000","last_unstale":"2023-10-07T23:21:25.815145+0000","last_undegraded":"2023-10-07T23:21:25.815145+0000","last_fullsized":"2023-10-07T23:21:25.815145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:37.712425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824534+0000","last_change":"2023-10-07T23:21:25.822953+0000","last_active":"2023-10-07T23:21:25.824534+0000","last_peered":"2023-10-07T23:21:25.824534+0000","last_clean":"2023-10-07T23:21:25.824534+0000","last_became_active":"2023-10-07T23:21:25.822567+0000","last_became_peered":"2023-10-07T23:21:25.822567+0000","last_unstale":"2023-10-07T23:21:25.824534+0000","last_undegraded":"2023-10-07T23:21:25.824534+0000","last_fullsized":"2023-10-07T23:21:25.824534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.818161+0000","last_change":"2023-10-07T23:21:25.810275+0000","last_active":"2023-10-07T23:21:25.818161+0000","last_peered":"2023-10-07T23:21:25.818161+0000","last_clean":"2023-10-07T23:21:25.818161+0000","last_became_active":"2023-10-07T23:21:25.808608+0000","last_became_peered":"2023-10-07T23:21:25.808608+0000","last_unstale":"2023-10-07T23:21:25.818161+0000","last_undegraded":"2023-10-07T23:21:25.818161+0000","last_fullsized":"2023-10-07T23:21:25.818161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815069+0000","last_change":"2023-10-07T23:21:25.810226+0000","last_active":"2023-10-07T23:21:25.815069+0000","last_peered":"2023-10-07T23:21:25.815069+0000","last_clean":"2023-10-07T23:21:25.815069+0000","last_became_active":"2023-10-07T23:21:25.809894+0000","last_became_peered":"2023-10-07T23:21:25.809894+0000","last_unstale":"2023-10-07T23:21:25.815069+0000","last_undegraded":"2023-10-07T23:21:25.815069+0000","last_fullsized":"2023-10-07T23:21:25.815069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820902+0000","last_change":"2023-10-07T23:21:25.811504+0000","last_active":"2023-10-07T23:21:25.820902+0000","last_peered":"2023-10-07T23:21:25.820902+0000","last_clean":"2023-10-07T23:21:25.820902+0000","last_became_active":"2023-10-07T23:21:25.811171+0000","last_became_peered":"2023-10-07T23:21:25.811171+0000","last_unstale":"2023-10-07T23:21:25.820902+0000","last_undegraded":"2023-10-07T23:21:25.820902+0000","last_fullsized":"2023-10-07T23:21:25.820902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813279+0000","last_change":"2023-10-07T23:21:25.811409+0000","last_active":"2023-10-07T23:21:25.813279+0000","last_peered":"2023-10-07T23:21:25.813279+0000","last_clean":"2023-10-07T23:21:25.813279+0000","last_became_active":"2023-10-07T23:21:25.811110+0000","last_became_peered":"2023-10-07T23:21:25.811110+0000","last_unstale":"2023-10-07T23:21:25.813279+0000","last_undegraded":"2023-10-07T23:21:25.813279+0000","last_fullsized":"2023-10-07T23:21:25.813279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831600+0000","last_change":"2023-10-07T23:21:25.823226+0000","last_active":"2023-10-07T23:21:25.831600+0000","last_peered":"2023-10-07T23:21:25.831600+0000","last_clean":"2023-10-07T23:21:25.831600+0000","last_became_active":"2023-10-07T23:21:25.822718+0000","last_became_peered":"2023-10-07T23:21:25.822718+0000","last_unstale":"2023-10-07T23:21:25.831600+0000","last_undegraded":"2023-10-07T23:21:25.831600+0000","last_fullsized":"2023-10-07T23:21:25.831600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816264+0000","last_change":"2023-10-07T23:21:25.808833+0000","last_active":"2023-10-07T23:21:25.816264+0000","last_peered":"2023-10-07T23:21:25.816264+0000","last_clean":"2023-10-07T23:21:25.816264+0000","last_became_active":"2023-10-07T23:21:25.808473+0000","last_became_peered":"2023-10-07T23:21:25.808473+0000","last_unstale":"2023-10-07T23:21:25.816264+0000","last_undegraded":"2023-10-07T23:21:25.816264+0000","last_fullsized":"2023-10-07T23:21:25.816264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822506+0000","last_change":"2023-10-07T23:21:25.815523+0000","last_active":"2023-10-07T23:21:25.822506+0000","last_peered":"2023-10-07T23:21:25.822506+0000","last_clean":"2023-10-07T23:21:25.822506+0000","last_became_active":"2023-10-07T23:21:25.814775+0000","last_became_peered":"2023-10-07T23:21:25.814775+0000","last_unstale":"2023-10-07T23:21:25.822506+0000","last_undegraded":"2023-10-07T23:21:25.822506+0000","last_fullsized":"2023-10-07T23:21:25.822506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823146+0000","last_change":"2023-10-07T23:21:25.818446+0000","last_active":"2023-10-07T23:21:25.823146+0000","last_peered":"2023-10-07T23:21:25.823146+0000","last_clean":"2023-10-07T23:21:25.823146+0000","last_became_active":"2023-10-07T23:21:25.818155+0000","last_became_peered":"2023-10-07T23:21:25.818155+0000","last_unstale":"2023-10-07T23:21:25.823146+0000","last_undegraded":"2023-10-07T23:21:25.823146+0000","last_fullsized":"2023-10-07T23:21:25.823146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822397+0000","last_change":"2023-10-07T23:21:25.815227+0000","last_active":"2023-10-07T23:21:25.822397+0000","last_peered":"2023-10-07T23:21:25.822397+0000","last_clean":"2023-10-07T23:21:25.822397+0000","last_became_active":"2023-10-07T23:21:25.814723+0000","last_became_peered":"2023-10-07T23:21:25.814723+0000","last_unstale":"2023-10-07T23:21:25.822397+0000","last_undegraded":"2023-10-07T23:21:25.822397+0000","last_fullsized":"2023-10-07T23:21:25.822397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810308+0000","last_change":"2023-10-07T23:21:25.808952+0000","last_active":"2023-10-07T23:21:25.810308+0000","last_peered":"2023-10-07T23:21:25.810308+0000","last_clean":"2023-10-07T23:21:25.810308+0000","last_became_active":"2023-10-07T23:21:25.808651+0000","last_became_peered":"2023-10-07T23:21:25.808651+0000","last_unstale":"2023-10-07T23:21:25.810308+0000","last_undegraded":"2023-10-07T23:21:25.810308+0000","last_fullsized":"2023-10-07T23:21:25.810308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816989+0000","last_change":"2023-10-07T23:21:25.816989+0000","last_active":"2023-10-07T23:21:25.816989+0000","last_peered":"2023-10-07T23:21:25.816989+0000","last_clean":"2023-10-07T23:21:25.816989+0000","last_became_active":"2023-10-07T23:21:25.816637+0000","last_became_peered":"2023-10-07T23:21:25.816637+0000","last_unstale":"2023-10-07T23:21:25.816989+0000","last_undegraded":"2023-10-07T23:21:25.816989+0000","last_fullsized":"2023-10-07T23:21:25.816989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:34.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824888+0000","last_change":"2023-10-07T23:21:25.817393+0000","last_active":"2023-10-07T23:21:25.824888+0000","last_peered":"2023-10-07T23:21:25.824888+0000","last_clean":"2023-10-07T23:21:25.824888+0000","last_became_active":"2023-10-07T23:21:25.816812+0000","last_became_peered":"2023-10-07T23:21:25.816812+0000","last_unstale":"2023-10-07T23:21:25.824888+0000","last_undegraded":"2023-10-07T23:21:25.824888+0000","last_fullsized":"2023-10-07T23:21:25.824888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831230+0000","last_change":"2023-10-07T23:21:25.820991+0000","last_active":"2023-10-07T23:21:25.831230+0000","last_peered":"2023-10-07T23:21:25.831230+0000","last_clean":"2023-10-07T23:21:25.831230+0000","last_became_active":"2023-10-07T23:21:25.820689+0000","last_became_peered":"2023-10-07T23:21:25.820689+0000","last_unstale":"2023-10-07T23:21:25.831230+0000","last_undegraded":"2023-10-07T23:21:25.831230+0000","last_fullsized":"2023-10-07T23:21:25.831230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822891+0000","last_change":"2023-10-07T23:21:25.818034+0000","last_active":"2023-10-07T23:21:25.822891+0000","last_peered":"2023-10-07T23:21:25.822891+0000","last_clean":"2023-10-07T23:21:25.822891+0000","last_became_active":"2023-10-07T23:21:25.817671+0000","last_became_peered":"2023-10-07T23:21:25.817671+0000","last_unstale":"2023-10-07T23:21:25.822891+0000","last_undegraded":"2023-10-07T23:21:25.822891+0000","last_fullsized":"2023-10-07T23:21:25.822891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816324+0000","last_change":"2023-10-07T23:21:25.814531+0000","last_active":"2023-10-07T23:21:25.816324+0000","last_peered":"2023-10-07T23:21:25.816324+0000","last_clean":"2023-10-07T23:21:25.816324+0000","last_became_active":"2023-10-07T23:21:25.814176+0000","last_became_peered":"2023-10-07T23:21:25.814176+0000","last_unstale":"2023-10-07T23:21:25.816324+0000","last_undegraded":"2023-10-07T23:21:25.816324+0000","last_fullsized":"2023-10-07T23:21:25.816324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:03.964344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819009+0000","last_change":"2023-10-07T23:21:25.815377+0000","last_active":"2023-10-07T23:21:25.819009+0000","last_peered":"2023-10-07T23:21:25.819009+0000","last_clean":"2023-10-07T23:21:25.819009+0000","last_became_active":"2023-10-07T23:21:25.815081+0000","last_became_peered":"2023-10-07T23:21:25.815081+0000","last_unstale":"2023-10-07T23:21:25.819009+0000","last_undegraded":"2023-10-07T23:21:25.819009+0000","last_fullsized":"2023-10-07T23:21:25.819009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816446+0000","last_change":"2023-10-07T23:21:25.809589+0000","last_active":"2023-10-07T23:21:25.816446+0000","last_peered":"2023-10-07T23:21:25.816446+0000","last_clean":"2023-10-07T23:21:25.816446+0000","last_became_active":"2023-10-07T23:21:25.809194+0000","last_became_peered":"2023-10-07T23:21:25.809194+0000","last_unstale":"2023-10-07T23:21:25.816446+0000","last_undegraded":"2023-10-07T23:21:25.816446+0000","last_fullsized":"2023-10-07T23:21:25.816446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:35.323397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.959548+0000","last_change":"2023-10-07T23:21:24.961412+0000","last_active":"2023-10-07T23:21:25.959548+0000","last_peered":"2023-10-07T23:21:25.959548+0000","last_clean":"2023-10-07T23:21:25.959548+0000","last_became_active":"2023-10-07T23:21:24.961027+0000","last_became_peered":"2023-10-07T23:21:24.961027+0000","last_unstale":"2023-10-07T23:21:25.959548+0000","last_undegraded":"2023-10-07T23:21:25.959548+0000","last_fullsized":"2023-10-07T23:21:25.959548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:13.180905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831277+0000","last_change":"2023-10-07T23:21:25.821055+0000","last_active":"2023-10-07T23:21:25.831277+0000","last_peered":"2023-10-07T23:21:25.831277+0000","last_clean":"2023-10-07T23:21:25.831277+0000","last_became_active":"2023-10-07T23:21:25.820736+0000","last_became_peered":"2023-10-07T23:21:25.820736+0000","last_unstale":"2023-10-07T23:21:25.831277+0000","last_undegraded":"2023-10-07T23:21:25.831277+0000","last_fullsized":"2023-10-07T23:21:25.831277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.952918+0000","last_change":"2023-10-07T23:21:24.946739+0000","last_active":"2023-10-07T23:21:25.952918+0000","last_peered":"2023-10-07T23:21:25.952918+0000","last_clean":"2023-10-07T23:21:25.952918+0000","last_became_active":"2023-10-07T23:21:24.946320+0000","last_became_peered":"2023-10-07T23:21:24.946320+0000","last_unstale":"2023-10-07T23:21:25.952918+0000","last_undegraded":"2023-10-07T23:21:25.952918+0000","last_fullsized":"2023-10-07T23:21:25.952918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810406+0000","last_change":"2023-10-07T23:21:24.955775+0000","last_active":"2023-10-07T23:21:25.810406+0000","last_peered":"2023-10-07T23:21:25.810406+0000","last_clean":"2023-10-07T23:21:25.810406+0000","last_became_active":"2023-10-07T23:21:24.955487+0000","last_became_peered":"2023-10-07T23:21:24.955487+0000","last_unstale":"2023-10-07T23:21:25.810406+0000","last_undegraded":"2023-10-07T23:21:25.810406+0000","last_fullsized":"2023-10-07T23:21:25.810406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825497+0000","last_change":"2023-10-07T23:21:25.822494+0000","last_active":"2023-10-07T23:21:25.825497+0000","last_peered":"2023-10-07T23:21:25.825497+0000","last_clean":"2023-10-07T23:21:25.825497+0000","last_became_active":"2023-10-07T23:21:25.821860+0000","last_became_peered":"2023-10-07T23:21:25.821860+0000","last_unstale":"2023-10-07T23:21:25.825497+0000","last_undegraded":"2023-10-07T23:21:25.825497+0000","last_fullsized":"2023-10-07T23:21:25.825497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.796432+0000","last_change":"2023-10-07T23:21:24.949505+0000","last_active":"2023-10-07T23:21:25.796432+0000","last_peered":"2023-10-07T23:21:25.796432+0000","last_clean":"2023-10-07T23:21:25.796432+0000","last_became_active":"2023-10-07T23:21:24.949195+0000","last_became_peered":"2023-10-07T23:21:24.949195+0000","last_unstale":"2023-10-07T23:21:25.796432+0000","last_undegraded":"2023-10-07T23:21:25.796432+0000","last_fullsized":"2023-10-07T23:21:25.796432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816941+0000","last_change":"2023-10-07T23:21:25.816941+0000","last_active":"2023-10-07T23:21:25.816941+0000","last_peered":"2023-10-07T23:21:25.816941+0000","last_clean":"2023-10-07T23:21:25.816941+0000","last_became_active":"2023-10-07T23:21:25.816658+0000","last_became_peered":"2023-10-07T23:21:25.816658+0000","last_unstale":"2023-10-07T23:21:25.816941+0000","last_undegraded":"2023-10-07T23:21:25.816941+0000","last_fullsized":"2023-10-07T23:21:25.816941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:20.455188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794906+0000","last_change":"2023-10-07T23:21:24.953996+0000","last_active":"2023-10-07T23:21:25.794906+0000","last_peered":"2023-10-07T23:21:25.794906+0000","last_clean":"2023-10-07T23:21:25.794906+0000","last_became_active":"2023-10-07T23:21:24.953684+0000","last_became_peered":"2023-10-07T23:21:24.953684+0000","last_unstale":"2023-10-07T23:21:25.794906+0000","last_undegraded":"2023-10-07T23:21:25.794906+0000","last_fullsized":"2023-10-07T23:21:25.794906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816295+0000","last_change":"2023-10-07T23:21:25.809600+0000","last_active":"2023-10-07T23:21:25.816295+0000","last_peered":"2023-10-07T23:21:25.816295+0000","last_clean":"2023-10-07T23:21:25.816295+0000","last_became_active":"2023-10-07T23:21:25.808699+0000","last_became_peered":"2023-10-07T23:21:25.808699+0000","last_unstale":"2023-10-07T23:21:25.816295+0000","last_undegraded":"2023-10-07T23:21:25.816295+0000","last_fullsized":"2023-10-07T23:21:25.816295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.793286+0000","last_change":"2023-10-07T23:21:24.952022+0000","last_active":"2023-10-07T23:21:25.793286+0000","last_peered":"2023-10-07T23:21:25.793286+0000","last_clean":"2023-10-07T23:21:25.793286+0000","last_became_active":"2023-10-07T23:21:24.951291+0000","last_became_peered":"2023-10-07T23:21:24.951291+0000","last_unstale":"2023-10-07T23:21:25.793286+0000","last_undegraded":"2023-10-07T23:21:25.793286+0000","last_fullsized":"2023-10-07T23:21:25.793286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823818+0000","last_change":"2023-10-07T23:21:25.816148+0000","last_active":"2023-10-07T23:21:25.823818+0000","last_peered":"2023-10-07T23:21:25.823818+0000","last_clean":"2023-10-07T23:21:25.823818+0000","last_became_active":"2023-10-07T23:21:25.815705+0000","last_became_peered":"2023-10-07T23:21:25.815705+0000","last_unstale":"2023-10-07T23:21:25.823818+0000","last_undegraded":"2023-10-07T23:21:25.823818+0000","last_fullsized":"2023-10-07T23:21:25.823818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:23.605003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954772+0000","last_change":"2023-10-07T23:21:24.954956+0000","last_active":"2023-10-07T23:21:25.954772+0000","last_peered":"2023-10-07T23:21:25.954772+0000","last_clean":"2023-10-07T23:21:25.954772+0000","last_became_active":"2023-10-07T23:21:24.954596+0000","last_became_peered":"2023-10-07T23:21:24.954596+0000","last_unstale":"2023-10-07T23:21:25.954772+0000","last_undegraded":"2023-10-07T23:21:25.954772+0000","last_fullsized":"2023-10-07T23:21:25.954772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:02.562726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816506+0000","last_change":"2023-10-07T23:21:25.809979+0000","last_active":"2023-10-07T23:21:25.816506+0000","last_peered":"2023-10-07T23:21:25.816506+0000","last_clean":"2023-10-07T23:21:25.816506+0000","last_became_active":"2023-10-07T23:21:25.809594+0000","last_became_peered":"2023-10-07T23:21:25.809594+0000","last_unstale":"2023-10-07T23:21:25.816506+0000","last_undegraded":"2023-10-07T23:21:25.816506+0000","last_fullsized":"2023-10-07T23:21:25.816506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:28.775950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.966427+0000","last_change":"2023-10-07T23:21:24.954704+0000","last_active":"2023-10-07T23:21:25.966427+0000","last_peered":"2023-10-07T23:21:25.966427+0000","last_clean":"2023-10-07T23:21:25.966427+0000","last_became_active":"2023-10-07T23:21:24.954416+0000","last_became_peered":"2023-10-07T23:21:24.954416+0000","last_unstale":"2023-10-07T23:21:25.966427+0000","last_undegraded":"2023-10-07T23:21:25.966427+0000","last_fullsized":"2023-10-07T23:21:25.966427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824095+0000","last_change":"2023-10-07T23:21:25.817538+0000","last_active":"2023-10-07T23:21:25.824095+0000","last_peered":"2023-10-07T23:21:25.824095+0000","last_clean":"2023-10-07T23:21:25.824095+0000","last_became_active":"2023-10-07T23:21:25.817209+0000","last_became_peered":"2023-10-07T23:21:25.817209+0000","last_unstale":"2023-10-07T23:21:25.824095+0000","last_undegraded":"2023-10-07T23:21:25.824095+0000","last_fullsized":"2023-10-07T23:21:25.824095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964849+0000","last_change":"2023-10-07T23:21:24.957350+0000","last_active":"2023-10-07T23:21:25.964849+0000","last_peered":"2023-10-07T23:21:25.964849+0000","last_clean":"2023-10-07T23:21:25.964849+0000","last_became_active":"2023-10-07T23:21:24.957009+0000","last_became_peered":"2023-10-07T23:21:24.957009+0000","last_unstale":"2023-10-07T23:21:25.964849+0000","last_undegraded":"2023-10-07T23:21:25.964849+0000","last_fullsized":"2023-10-07T23:21:25.964849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:58.493058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823798+0000","last_change":"2023-10-07T23:21:25.814807+0000","last_active":"2023-10-07T23:21:25.823798+0000","last_peered":"2023-10-07T23:21:25.823798+0000","last_clean":"2023-10-07T23:21:25.823798+0000","last_became_active":"2023-10-07T23:21:25.813589+0000","last_became_peered":"2023-10-07T23:21:25.813589+0000","last_unstale":"2023-10-07T23:21:25.823798+0000","last_undegraded":"2023-10-07T23:21:25.823798+0000","last_fullsized":"2023-10-07T23:21:25.823798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797030+0000","last_change":"2023-10-07T23:21:24.957704+0000","last_active":"2023-10-07T23:21:25.797030+0000","last_peered":"2023-10-07T23:21:25.797030+0000","last_clean":"2023-10-07T23:21:25.797030+0000","last_became_active":"2023-10-07T23:21:24.957447+0000","last_became_peered":"2023-10-07T23:21:24.957447+0000","last_unstale":"2023-10-07T23:21:25.797030+0000","last_undegraded":"2023-10-07T23:21:25.797030+0000","last_fullsized":"2023-10-07T23:21:25.797030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824121+0000","last_change":"2023-10-07T23:21:25.817566+0000","last_active":"2023-10-07T23:21:25.824121+0000","last_peered":"2023-10-07T23:21:25.824121+0000","last_clean":"2023-10-07T23:21:25.824121+0000","last_became_active":"2023-10-07T23:21:25.816898+0000","last_became_peered":"2023-10-07T23:21:25.816898+0000","last_unstale":"2023-10-07T23:21:25.824121+0000","last_undegraded":"2023-10-07T23:21:25.824121+0000","last_fullsized":"2023-10-07T23:21:25.824121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794453+0000","last_change":"2023-10-07T23:21:24.948051+0000","last_active":"2023-10-07T23:21:25.794453+0000","last_peered":"2023-10-07T23:21:25.794453+0000","last_clean":"2023-10-07T23:21:25.794453+0000","last_became_active":"2023-10-07T23:21:24.947742+0000","last_became_peered":"2023-10-07T23:21:24.947742+0000","last_unstale":"2023-10-07T23:21:25.794453+0000","last_undegraded":"2023-10-07T23:21:25.794453+0000","last_fullsized":"2023-10-07T23:21:25.794453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819027+0000","last_change":"2023-10-07T23:21:25.815665+0000","last_active":"2023-10-07T23:21:25.819027+0000","last_peered":"2023-10-07T23:21:25.819027+0000","last_clean":"2023-10-07T23:21:25.819027+0000","last_became_active":"2023-10-07T23:21:25.815286+0000","last_became_peered":"2023-10-07T23:21:25.815286+0000","last_unstale":"2023-10-07T23:21:25.819027+0000","last_undegraded":"2023-10-07T23:21:25.819027+0000","last_fullsized":"2023-10-07T23:21:25.819027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.960851+0000","last_change":"2023-10-07T23:21:24.952011+0000","last_active":"2023-10-07T23:21:25.960851+0000","last_peered":"2023-10-07T23:21:25.960851+0000","last_clean":"2023-10-07T23:21:25.960851+0000","last_became_active":"2023-10-07T23:21:24.951691+0000","last_became_peered":"2023-10-07T23:21:24.951691+0000","last_unstale":"2023-10-07T23:21:25.960851+0000","last_undegraded":"2023-10-07T23:21:25.960851+0000","last_fullsized":"2023-10-07T23:21:25.960851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823976+0000","last_change":"2023-10-07T23:21:25.817054+0000","last_active":"2023-10-07T23:21:25.823976+0000","last_peered":"2023-10-07T23:21:25.823976+0000","last_clean":"2023-10-07T23:21:25.823976+0000","last_became_active":"2023-10-07T23:21:25.816667+0000","last_became_peered":"2023-10-07T23:21:25.816667+0000","last_unstale":"2023-10-07T23:21:25.823976+0000","last_undegraded":"2023-10-07T23:21:25.823976+0000","last_fullsized":"2023-10-07T23:21:25.823976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.961478+0000","last_change":"2023-10-07T23:21:24.958951+0000","last_active":"2023-10-07T23:21:25.961478+0000","last_peered":"2023-10-07T23:21:25.961478+0000","last_clean":"2023-10-07T23:21:25.961478+0000","last_became_active":"2023-10-07T23:21:24.958632+0000","last_became_peered":"2023-10-07T23:21:24.958632+0000","last_unstale":"2023-10-07T23:21:25.961478+0000","last_undegraded":"2023-10-07T23:21:25.961478+0000","last_fullsized":"2023-10-07T23:21:25.961478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:29.730287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823870+0000","last_change":"2023-10-07T23:21:25.816914+0000","last_active":"2023-10-07T23:21:25.823870+0000","last_peered":"2023-10-07T23:21:25.823870+0000","last_clean":"2023-10-07T23:21:25.823870+0000","last_became_active":"2023-10-07T23:21:25.816515+0000","last_became_peered":"2023-10-07T23:21:25.816515+0000","last_unstale":"2023-10-07T23:21:25.823870+0000","last_undegraded":"2023-10-07T23:21:25.823870+0000","last_fullsized":"2023-10-07T23:21:25.823870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.955373+0000","last_change":"2023-10-07T23:21:24.956912+0000","last_active":"2023-10-07T23:21:25.955373+0000","last_peered":"2023-10-07T23:21:25.955373+0000","last_clean":"2023-10-07T23:21:25.955373+0000","last_became_active":"2023-10-07T23:21:24.956588+0000","last_became_peered":"2023-10-07T23:21:24.956588+0000","last_unstale":"2023-10-07T23:21:25.955373+0000","last_undegraded":"2023-10-07T23:21:25.955373+0000","last_fullsized":"2023-10-07T23:21:25.955373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824203+0000","last_change":"2023-10-07T23:21:25.818653+0000","last_active":"2023-10-07T23:21:25.824203+0000","last_peered":"2023-10-07T23:21:25.824203+0000","last_clean":"2023-10-07T23:21:25.824203+0000","last_became_active":"2023-10-07T23:21:25.817833+0000","last_became_peered":"2023-10-07T23:21:25.817833+0000","last_unstale":"2023-10-07T23:21:25.824203+0000","last_undegraded":"2023-10-07T23:21:25.824203+0000","last_fullsized":"2023-10-07T23:21:25.824203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.800876+0000","last_change":"2023-10-07T23:21:24.948082+0000","last_active":"2023-10-07T23:21:25.800876+0000","last_peered":"2023-10-07T23:21:25.800876+0000","last_clean":"2023-10-07T23:21:25.800876+0000","last_became_active":"2023-10-07T23:21:24.947759+0000","last_became_peered":"2023-10-07T23:21:24.947759+0000","last_unstale":"2023-10-07T23:21:25.800876+0000","last_undegraded":"2023-10-07T23:21:25.800876+0000","last_fullsized":"2023-10-07T23:21:25.800876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:11.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822945+0000","last_change":"2023-10-07T23:21:25.817833+0000","last_active":"2023-10-07T23:21:25.822945+0000","last_peered":"2023-10-07T23:21:25.822945+0000","last_clean":"2023-10-07T23:21:25.822945+0000","last_became_active":"2023-10-07T23:21:25.817547+0000","last_became_peered":"2023-10-07T23:21:25.817547+0000","last_unstale":"2023-10-07T23:21:25.822945+0000","last_undegraded":"2023-10-07T23:21:25.822945+0000","last_fullsized":"2023-10-07T23:21:25.822945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.807532+0000","last_change":"2023-10-07T23:21:24.956344+0000","last_active":"2023-10-07T23:21:25.807532+0000","last_peered":"2023-10-07T23:21:25.807532+0000","last_clean":"2023-10-07T23:21:25.807532+0000","last_became_active":"2023-10-07T23:21:24.955974+0000","last_became_peered":"2023-10-07T23:21:24.955974+0000","last_unstale":"2023-10-07T23:21:25.807532+0000","last_undegraded":"2023-10-07T23:21:25.807532+0000","last_fullsized":"2023-10-07T23:21:25.807532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.816194+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.813965+0000","last_became_peered":"2023-10-07T23:21:25.813965+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802932+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:21:25.802932+0000","last_peered":"2023-10-07T23:21:25.802932+0000","last_clean":"2023-10-07T23:21:25.802932+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:21:25.802932+0000","last_undegraded":"2023-10-07T23:21:25.802932+0000","last_fullsized":"2023-10-07T23:21:25.802932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.792857+0000","last_change":"2023-10-07T23:21:24.953534+0000","last_active":"2023-10-07T23:21:25.792857+0000","last_peered":"2023-10-07T23:21:25.792857+0000","last_clean":"2023-10-07T23:21:25.792857+0000","last_became_active":"2023-10-07T23:21:24.953202+0000","last_became_peered":"2023-10-07T23:21:24.953202+0000","last_unstale":"2023-10-07T23:21:25.792857+0000","last_undegraded":"2023-10-07T23:21:25.792857+0000","last_fullsized":"2023-10-07T23:21:25.792857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824763+0000","last_change":"2023-10-07T23:21:25.816874+0000","last_active":"2023-10-07T23:21:25.824763+0000","last_peered":"2023-10-07T23:21:25.824763+0000","last_clean":"2023-10-07T23:21:25.824763+0000","last_became_active":"2023-10-07T23:21:25.816098+0000","last_became_peered":"2023-10-07T23:21:25.816098+0000","last_unstale":"2023-10-07T23:21:25.824763+0000","last_undegraded":"2023-10-07T23:21:25.824763+0000","last_fullsized":"2023-10-07T23:21:25.824763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802840+0000","last_change":"2023-10-07T23:21:24.962711+0000","last_active":"2023-10-07T23:21:25.802840+0000","last_peered":"2023-10-07T23:21:25.802840+0000","last_clean":"2023-10-07T23:21:25.802840+0000","last_became_active":"2023-10-07T23:21:24.962404+0000","last_became_peered":"2023-10-07T23:21:24.962404+0000","last_unstale":"2023-10-07T23:21:25.802840+0000","last_undegraded":"2023-10-07T23:21:25.802840+0000","last_fullsized":"2023-10-07T23:21:25.802840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:02.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819414+0000","last_change":"2023-10-07T23:21:25.809376+0000","last_active":"2023-10-07T23:21:25.819414+0000","last_peered":"2023-10-07T23:21:25.819414+0000","last_clean":"2023-10-07T23:21:25.819414+0000","last_became_active":"2023-10-07T23:21:25.809055+0000","last_became_peered":"2023-10-07T23:21:25.809055+0000","last_unstale":"2023-10-07T23:21:25.819414+0000","last_undegraded":"2023-10-07T23:21:25.819414+0000","last_fullsized":"2023-10-07T23:21:25.819414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806772+0000","last_change":"2023-10-07T23:21:24.960450+0000","last_active":"2023-10-07T23:21:25.806772+0000","last_peered":"2023-10-07T23:21:25.806772+0000","last_clean":"2023-10-07T23:21:25.806772+0000","last_became_active":"2023-10-07T23:21:24.960068+0000","last_became_peered":"2023-10-07T23:21:24.960068+0000","last_unstale":"2023-10-07T23:21:25.806772+0000","last_undegraded":"2023-10-07T23:21:25.806772+0000","last_fullsized":"2023-10-07T23:21:25.806772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815004+0000","last_change":"2023-10-07T23:21:25.815004+0000","last_active":"2023-10-07T23:21:25.815004+0000","last_peered":"2023-10-07T23:21:25.815004+0000","last_clean":"2023-10-07T23:21:25.815004+0000","last_became_active":"2023-10-07T23:21:25.814688+0000","last_became_peered":"2023-10-07T23:21:25.814688+0000","last_unstale":"2023-10-07T23:21:25.815004+0000","last_undegraded":"2023-10-07T23:21:25.815004+0000","last_fullsized":"2023-10-07T23:21:25.815004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:46.190235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797147+0000","last_change":"2023-10-07T23:21:24.952189+0000","last_active":"2023-10-07T23:21:25.797147+0000","last_peered":"2023-10-07T23:21:25.797147+0000","last_clean":"2023-10-07T23:21:25.797147+0000","last_became_active":"2023-10-07T23:21:24.951731+0000","last_became_peered":"2023-10-07T23:21:24.951731+0000","last_unstale":"2023-10-07T23:21:25.797147+0000","last_undegraded":"2023-10-07T23:21:25.797147+0000","last_fullsized":"2023-10-07T23:21:25.797147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823922+0000","last_change":"2023-10-07T23:21:25.822875+0000","last_active":"2023-10-07T23:21:25.823922+0000","last_peered":"2023-10-07T23:21:25.823922+0000","last_clean":"2023-10-07T23:21:25.823922+0000","last_became_active":"2023-10-07T23:21:25.822536+0000","last_became_peered":"2023-10-07T23:21:25.822536+0000","last_unstale":"2023-10-07T23:21:25.823922+0000","last_undegraded":"2023-10-07T23:21:25.823922+0000","last_fullsized":"2023-10-07T23:21:25.823922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:47.179786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.960593+0000","last_change":"2023-10-07T23:21:24.952562+0000","last_active":"2023-10-07T23:21:24.960593+0000","last_peered":"2023-10-07T23:21:24.960593+0000","last_clean":"2023-10-07T23:21:24.960593+0000","last_became_active":"2023-10-07T23:21:24.952071+0000","last_became_peered":"2023-10-07T23:21:24.952071+0000","last_unstale":"2023-10-07T23:21:24.960593+0000","last_undegraded":"2023-10-07T23:21:24.960593+0000","last_fullsized":"2023-10-07T23:21:24.960593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:17.603304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822867+0000","last_change":"2023-10-07T23:21:25.818214+0000","last_active":"2023-10-07T23:21:25.822867+0000","last_peered":"2023-10-07T23:21:25.822867+0000","last_clean":"2023-10-07T23:21:25.822867+0000","last_became_active":"2023-10-07T23:21:25.817888+0000","last_became_peered":"2023-10-07T23:21:25.817888+0000","last_unstale":"2023-10-07T23:21:25.822867+0000","last_undegraded":"2023-10-07T23:21:25.822867+0000","last_fullsized":"2023-10-07T23:21:25.822867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.804473+0000","last_change":"2023-10-07T23:21:24.962199+0000","last_active":"2023-10-07T23:21:25.804473+0000","last_peered":"2023-10-07T23:21:25.804473+0000","last_clean":"2023-10-07T23:21:25.804473+0000","last_became_active":"2023-10-07T23:21:24.961778+0000","last_became_peered":"2023-10-07T23:21:24.961778+0000","last_unstale":"2023-10-07T23:21:25.804473+0000","last_undegraded":"2023-10-07T23:21:25.804473+0000","last_fullsized":"2023-10-07T23:21:25.804473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815292+0000","last_change":"2023-10-07T23:21:25.815292+0000","last_active":"2023-10-07T23:21:25.815292+0000","last_peered":"2023-10-07T23:21:25.815292+0000","last_clean":"2023-10-07T23:21:25.815292+0000","last_became_active":"2023-10-07T23:21:25.814294+0000","last_became_peered":"2023-10-07T23:21:25.814294+0000","last_unstale":"2023-10-07T23:21:25.815292+0000","last_undegraded":"2023-10-07T23:21:25.815292+0000","last_fullsized":"2023-10-07T23:21:25.815292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:41.638201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.952777+0000","last_change":"2023-10-07T23:21:24.948131+0000","last_active":"2023-10-07T23:21:24.952777+0000","last_peered":"2023-10-07T23:21:24.952777+0000","last_clean":"2023-10-07T23:21:24.952777+0000","last_became_active":"2023-10-07T23:21:24.947831+0000","last_became_peered":"2023-10-07T23:21:24.947831+0000","last_unstale":"2023-10-07T23:21:24.952777+0000","last_undegraded":"2023-10-07T23:21:24.952777+0000","last_fullsized":"2023-10-07T23:21:24.952777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:23.995859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825605+0000","last_change":"2023-10-07T23:21:25.823117+0000","last_active":"2023-10-07T23:21:25.825605+0000","last_peered":"2023-10-07T23:21:25.825605+0000","last_clean":"2023-10-07T23:21:25.825605+0000","last_became_active":"2023-10-07T23:21:25.822814+0000","last_became_peered":"2023-10-07T23:21:25.822814+0000","last_unstale":"2023-10-07T23:21:25.825605+0000","last_undegraded":"2023-10-07T23:21:25.825605+0000","last_fullsized":"2023-10-07T23:21:25.825605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954544+0000","last_change":"2023-10-07T23:21:24.957129+0000","last_active":"2023-10-07T23:21:25.954544+0000","last_peered":"2023-10-07T23:21:25.954544+0000","last_clean":"2023-10-07T23:21:25.954544+0000","last_became_active":"2023-10-07T23:21:24.956815+0000","last_became_peered":"2023-10-07T23:21:24.956815+0000","last_unstale":"2023-10-07T23:21:25.954544+0000","last_undegraded":"2023-10-07T23:21:25.954544+0000","last_fullsized":"2023-10-07T23:21:25.954544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823723+0000","last_change":"2023-10-07T23:21:25.814482+0000","last_active":"2023-10-07T23:21:25.823723+0000","last_peered":"2023-10-07T23:21:25.823723+0000","last_clean":"2023-10-07T23:21:25.823723+0000","last_became_active":"2023-10-07T23:21:25.814126+0000","last_became_peered":"2023-10-07T23:21:25.814126+0000","last_unstale":"2023-10-07T23:21:25.823723+0000","last_undegraded":"2023-10-07T23:21:25.823723+0000","last_fullsized":"2023-10-07T23:21:25.823723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802684+0000","last_change":"2023-10-07T23:21:24.960263+0000","last_active":"2023-10-07T23:21:25.802684+0000","last_peered":"2023-10-07T23:21:25.802684+0000","last_clean":"2023-10-07T23:21:25.802684+0000","last_became_active":"2023-10-07T23:21:24.959984+0000","last_became_peered":"2023-10-07T23:21:24.959984+0000","last_unstale":"2023-10-07T23:21:25.802684+0000","last_undegraded":"2023-10-07T23:21:25.802684+0000","last_fullsized":"2023-10-07T23:21:25.802684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:34.309999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.821174+0000","last_change":"2023-10-07T23:21:25.819404+0000","last_active":"2023-10-07T23:21:25.821174+0000","last_peered":"2023-10-07T23:21:25.821174+0000","last_clean":"2023-10-07T23:21:25.821174+0000","last_became_active":"2023-10-07T23:21:25.818658+0000","last_became_peered":"2023-10-07T23:21:25.818658+0000","last_unstale":"2023-10-07T23:21:25.821174+0000","last_undegraded":"2023-10-07T23:21:25.821174+0000","last_fullsized":"2023-10-07T23:21:25.821174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.961205+0000","last_change":"2023-10-07T23:21:24.958141+0000","last_active":"2023-10-07T23:21:24.961205+0000","last_peered":"2023-10-07T23:21:24.961205+0000","last_clean":"2023-10-07T23:21:24.961205+0000","last_became_active":"2023-10-07T23:21:24.957791+0000","last_became_peered":"2023-10-07T23:21:24.957791+0000","last_unstale":"2023-10-07T23:21:24.961205+0000","last_undegraded":"2023-10-07T23:21:24.961205+0000","last_fullsized":"2023-10-07T23:21:24.961205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:01.467025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825628+0000","last_change":"2023-10-07T23:21:25.823525+0000","last_active":"2023-10-07T23:21:25.825628+0000","last_peered":"2023-10-07T23:21:25.825628+0000","last_clean":"2023-10-07T23:21:25.825628+0000","last_became_active":"2023-10-07T23:21:25.822193+0000","last_became_peered":"2023-10-07T23:21:25.822193+0000","last_unstale":"2023-10-07T23:21:25.825628+0000","last_undegraded":"2023-10-07T23:21:25.825628+0000","last_fullsized":"2023-10-07T23:21:25.825628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-07T23:21:24.961237+0000","last_change":"2023-10-07T23:21:24.957341+0000","last_active":"2023-10-07T23:21:24.961237+0000","last_peered":"2023-10-07T23:21:24.961237+0000","last_clean":"2023-10-07T23:21:24.961237+0000","last_became_active":"2023-10-07T23:21:24.957059+0000","last_became_peered":"2023-10-07T23:21:24.957059+0000","last_unstale":"2023-10-07T23:21:24.961237+0000","last_undegraded":"2023-10-07T23:21:24.961237+0000","last_fullsized":"2023-10-07T23:21:24.961237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.702939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825528+0000","last_change":"2023-10-07T23:21:25.822814+0000","last_active":"2023-10-07T23:21:25.825528+0000","last_peered":"2023-10-07T23:21:25.825528+0000","last_clean":"2023-10-07T23:21:25.825528+0000","last_became_active":"2023-10-07T23:21:25.822297+0000","last_became_peered":"2023-10-07T23:21:25.822297+0000","last_unstale":"2023-10-07T23:21:25.825528+0000","last_undegraded":"2023-10-07T23:21:25.825528+0000","last_fullsized":"2023-10-07T23:21:25.825528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954772+0000","last_change":"2023-10-07T23:21:24.961807+0000","last_active":"2023-10-07T23:21:25.954772+0000","last_peered":"2023-10-07T23:21:25.954772+0000","last_clean":"2023-10-07T23:21:25.954772+0000","last_became_active":"2023-10-07T23:21:24.961451+0000","last_became_peered":"2023-10-07T23:21:24.961451+0000","last_unstale":"2023-10-07T23:21:25.954772+0000","last_undegraded":"2023-10-07T23:21:25.954772+0000","last_fullsized":"2023-10-07T23:21:25.954772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813287+0000","last_change":"2023-10-07T23:21:25.811192+0000","last_active":"2023-10-07T23:21:25.813287+0000","last_peered":"2023-10-07T23:21:25.813287+0000","last_clean":"2023-10-07T23:21:25.813287+0000","last_became_active":"2023-10-07T23:21:25.810877+0000","last_became_peered":"2023-10-07T23:21:25.810877+0000","last_unstale":"2023-10-07T23:21:25.813287+0000","last_undegraded":"2023-10-07T23:21:25.813287+0000","last_fullsized":"2023-10-07T23:21:25.813287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816476+0000","last_change":"2023-10-07T23:21:25.811521+0000","last_active":"2023-10-07T23:21:25.816476+0000","last_peered":"2023-10-07T23:21:25.816476+0000","last_clean":"2023-10-07T23:21:25.816476+0000","last_became_active":"2023-10-07T23:21:25.811256+0000","last_became_peered":"2023-10-07T23:21:25.811256+0000","last_unstale":"2023-10-07T23:21:25.816476+0000","last_undegraded":"2023-10-07T23:21:25.816476+0000","last_fullsized":"2023-10-07T23:21:25.816476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954460+0000","last_change":"2023-10-07T23:21:24.962057+0000","last_active":"2023-10-07T23:21:25.954460+0000","last_peered":"2023-10-07T23:21:25.954460+0000","last_clean":"2023-10-07T23:21:25.954460+0000","last_became_active":"2023-10-07T23:21:24.961540+0000","last_became_peered":"2023-10-07T23:21:24.961540+0000","last_unstale":"2023-10-07T23:21:25.954460+0000","last_undegraded":"2023-10-07T23:21:25.954460+0000","last_fullsized":"2023-10-07T23:21:25.954460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.817631+0000","last_change":"2023-10-07T23:21:25.812184+0000","last_active":"2023-10-07T23:21:25.817631+0000","last_peered":"2023-10-07T23:21:25.817631+0000","last_clean":"2023-10-07T23:21:25.817631+0000","last_became_active":"2023-10-07T23:21:25.811844+0000","last_became_peered":"2023-10-07T23:21:25.811844+0000","last_unstale":"2023-10-07T23:21:25.817631+0000","last_undegraded":"2023-10-07T23:21:25.817631+0000","last_fullsized":"2023-10-07T23:21:25.817631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954389+0000","last_change":"2023-10-07T23:21:24.961653+0000","last_active":"2023-10-07T23:21:25.954389+0000","last_peered":"2023-10-07T23:21:25.954389+0000","last_clean":"2023-10-07T23:21:25.954389+0000","last_became_active":"2023-10-07T23:21:24.961362+0000","last_became_peered":"2023-10-07T23:21:24.961362+0000","last_unstale":"2023-10-07T23:21:25.954389+0000","last_undegraded":"2023-10-07T23:21:25.954389+0000","last_fullsized":"2023-10-07T23:21:25.954389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:27.376709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823793+0000","last_change":"2023-10-07T23:21:25.813813+0000","last_active":"2023-10-07T23:21:25.823793+0000","last_peered":"2023-10-07T23:21:25.823793+0000","last_clean":"2023-10-07T23:21:25.823793+0000","last_became_active":"2023-10-07T23:21:25.813500+0000","last_became_peered":"2023-10-07T23:21:25.813500+0000","last_unstale":"2023-10-07T23:21:25.823793+0000","last_undegraded":"2023-10-07T23:21:25.823793+0000","last_fullsized":"2023-10-07T23:21:25.823793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.965038+0000","last_change":"2023-10-07T23:21:24.959465+0000","last_active":"2023-10-07T23:21:25.965038+0000","last_peered":"2023-10-07T23:21:25.965038+0000","last_clean":"2023-10-07T23:21:25.965038+0000","last_became_active":"2023-10-07T23:21:24.957700+0000","last_became_peered":"2023-10-07T23:21:24.957700+0000","last_unstale":"2023-10-07T23:21:25.965038+0000","last_undegraded":"2023-10-07T23:21:25.965038+0000","last_fullsized":"2023-10-07T23:21:25.965038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824672+0000","last_change":"2023-10-07T23:21:25.816098+0000","last_active":"2023-10-07T23:21:25.824672+0000","last_peered":"2023-10-07T23:21:25.824672+0000","last_clean":"2023-10-07T23:21:25.824672+0000","last_became_active":"2023-10-07T23:21:25.815533+0000","last_became_peered":"2023-10-07T23:21:25.815533+0000","last_unstale":"2023-10-07T23:21:25.824672+0000","last_undegraded":"2023-10-07T23:21:25.824672+0000","last_fullsized":"2023-10-07T23:21:25.824672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964792+0000","last_change":"2023-10-07T23:21:24.960874+0000","last_active":"2023-10-07T23:21:25.964792+0000","last_peered":"2023-10-07T23:21:25.964792+0000","last_clean":"2023-10-07T23:21:25.964792+0000","last_became_active":"2023-10-07T23:21:24.960568+0000","last_became_peered":"2023-10-07T23:21:24.960568+0000","last_unstale":"2023-10-07T23:21:25.964792+0000","last_undegraded":"2023-10-07T23:21:25.964792+0000","last_fullsized":"2023-10-07T23:21:25.964792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298088,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073752,"statfs":{"total":96636764160,"available":96331522048,"internally_reserved":0,"allocated":819200,"data_stored":665265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":75879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":74433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":233472,"data_stored":74469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301612,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070228,"statfs":{"total":96636764160,"available":96327913472,"internally_reserved":0,"allocated":233472,"data_stored":75879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301600,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070240,"statfs":{"total":96636764160,"available":96327925760,"internally_reserved":0,"allocated":221184,"data_stored":74343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302192,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069648,"statfs":{"total":96636764160,"available":96327319552,"internally_reserved":0,"allocated":827392,"data_stored":665321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301608,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070232,"statfs":{"total":96636764160,"available":96327917568,"internally_reserved":0,"allocated":229376,"data_stored":74503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:21:29.301 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T23:21:29.301 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-07T23:21:29.301 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-07T23:21:29.301 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-07T23:21:29.301 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-07T23:21:29.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 Processor -- start 2023-10-07T23:21:29.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 -- start start 2023-10-07T23:21:29.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b4068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f13b4158b00 0x7f13b406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b406d750 con 0x7f13b40c7800 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40c8b80 con 0x7f13b4158b00 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b9c01700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c7f70 con 0x7f13b4068530 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b2ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f13b40c7800 0x7f13b41583f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60438/0 (socket says 172.21.15.57:60438) 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b2ffd700 1 -- 172.21.15.57:0/2641114027 learned_addr learned my addr 172.21.15.57:0/2641114027 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b3fff700 1 --2- 172.21.15.57:0/2641114027 >> v2:172.21.15.154:3300/0 conn(0x7f13b4158b00 0x7f13b406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b37fe700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b4068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218761376 0 0) 0x7f13a8001110 con 0x7f13b40c7800 2023-10-07T23:21:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.406+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f139c003570 con 0x7f13b40c7800 2023-10-07T23:21:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b3fff700 1 -- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 msgr2=0x7f13b4068900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619102468 0 0) 0x7f139c003570 con 0x7f13b40c7800 2023-10-07T23:21:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b3fff700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b4068900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b3fff700 1 -- 172.21.15.57:0/2641114027 >> v1:172.21.15.57:6789/0 conn(0x7f13b40c7800 legacy=0x7f13b41583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b3fff700 1 -- 172.21.15.57:0/2641114027 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40c8250 con 0x7f13b4158b00 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b37fe700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b4068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b3fff700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b4158b00 0x7f13b406d210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00af70 tx=0x7f13ac00a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f764bce80fef3e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00f070 con 0x7f13b4158b00 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac00a0d0 con 0x7f13b4158b00 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b27fc700 1 -- 172.21.15.57:0/2641114027 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8001600 con 0x7f13b4158b00 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 -- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b4158b00 msgr2=0x7f13b406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b4158b00 0x7f13b406d210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00af70 tx=0x7f13ac00a890 comp rx=0 tx=0).stop 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 -- 172.21.15.57:0/2641114027 shutdown_connections 2023-10-07T23:21:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b4158b00 0x7f13b406d210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 --2- 172.21.15.57:0/2641114027 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b4068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 -- 172.21.15.57:0/2641114027 >> 172.21.15.57:0/2641114027 conn(0x7f13b40cce30 msgr2=0x7f13b40c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 -- 172.21.15.57:0/2641114027 shutdown_connections 2023-10-07T23:21:29.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 -- 172.21.15.57:0/2641114027 wait complete. 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.410+0000 7f13b9c01700 1 Processor -- start 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- start start 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b40c7800 0x7f13b4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4153700 con 0x7f13b4158b00 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41570d0 con 0x7f13b40c7800 2023-10-07T23:21:29.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41573b0 con 0x7f13b4068530 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58242/0 (socket says 172.21.15.57:58242) 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 -- 172.21.15.57:0/505671271 learned_addr learned my addr 172.21.15.57:0/505671271 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b2ffd700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b40c7800 0x7f13b4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b37fe700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 -- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 msgr2=0x7f13b40c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b40c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 -- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b40c7800 msgr2=0x7f13b4152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b40c7800 0x7f13b4152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 -- 172.21.15.57:0/505671271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009420 con 0x7f13b4158b00 2023-10-07T23:21:29.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b3fff700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00a8d0 tx=0x7f13ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0096f0 con 0x7f13b4158b00 2023-10-07T23:21:29.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac009870 con 0x7f13b4158b00 2023-10-07T23:21:29.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac00ab30 con 0x7f13b4158b00 2023-10-07T23:21:29.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac009c30 con 0x7f13b4158b00 2023-10-07T23:21:29.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.414+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4157970 con 0x7f13b4158b00 2023-10-07T23:21:29.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b40711c0 con 0x7f13b4158b00 2023-10-07T23:21:29.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f13ac01a070 con 0x7f13b4158b00 2023-10-07T23:21:29.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f139b7fe700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 0x7f13a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f13ac07d9a0 con 0x7f13b4158b00 2023-10-07T23:21:29.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f13b37fe700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 0x7f13a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.418+0000 7f13b37fe700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 0x7f13a0062a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009f40 tx=0x7f13a40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.426+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f13ac04df90 con 0x7f13b4158b00 2023-10-07T23:21:29.514 INFO:tasks.ceph.mds.b.smithi154.stderr:2023-10-07T23:21:29.512+0000 7fa214b7d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:21:29.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.626+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f13b41588e0 con 0x7f13b4158b00 2023-10-07T23:21:29.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.634+0000 7f139b7fe700 1 -- 172.21.15.57:0/505671271 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154138 (secure 0 0 0) 0x7f13ac0515b0 con 0x7f13b4158b00 2023-10-07T23:21:29.640 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 msgr2=0x7f13a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 0x7f13a0062a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009f40 tx=0x7f13a40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 msgr2=0x7f13b41531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00a8d0 tx=0x7f13ac018000 comp rx=0 tx=0).stop 2023-10-07T23:21:29.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 shutdown_connections 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13b4158b00 0x7f13b41531c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13a0060550 0x7f13a0062a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13b40c7800 0x7f13b4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 --2- 172.21.15.57:0/505671271 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13b4068530 0x7f13b40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 >> 172.21.15.57:0/505671271 conn(0x7f13b40cce30 msgr2=0x7f13b4070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 shutdown_connections 2023-10-07T23:21:29.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.638+0000 7f13b9c01700 1 -- 172.21.15.57:0/505671271 wait complete. 2023-10-07T23:21:29.655 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":33383},{"type":"v1","addr":"172.21.15.154:6833","nonce":33383}]},"active_addr":"172.21.15.154:6833/33383","active_change":"2023-10-07T23:21:05.050623+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1098283269}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":2175543758}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1896752135}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1961047133}]}]} 2023-10-07T23:21:29.659 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-07T23:21:29.659 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-07T23:21:29.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:29.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.758+0000 7feded0c2700 1 Processor -- start 2023-10-07T23:21:29.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.758+0000 7feded0c2700 1 -- start start 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158be0 0x7fede80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fede8158fb0 0x7fede80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede806b3c0 con 0x7fede806b850 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede806a7b0 con 0x7fede8158fb0 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede806aa90 con 0x7fede8158be0 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fede8158fb0 0x7fede80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fede8158fb0 0x7fede80ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58244/0 (socket says 172.21.15.57:58244) 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 -- 172.21.15.57:0/2094207653 learned_addr learned my addr 172.21.15.57:0/2094207653 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:29.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede7fff700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158be0 0x7fede80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 -- 172.21.15.57:0/2094207653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158be0 msgr2=0x7fede80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158be0 0x7fede80ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 -- 172.21.15.57:0/2094207653 >> v1:172.21.15.57:6789/0 conn(0x7fede806b850 legacy=0x7fede80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede6ffd700 1 -- 172.21.15.57:0/2094207653 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956282535 0 0) 0x7fede806b3c0 con 0x7fede806b850 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 -- 172.21.15.57:0/2094207653 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede806ad70 con 0x7fede8158fb0 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede77fe700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158fb0 0x7fede80ce6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009f20 tx=0x7fedd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b40a51bf534584 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede6ffd700 1 -- 172.21.15.57:0/2094207653 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000e070 con 0x7fede8158fb0 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede6ffd700 1 -- 172.21.15.57:0/2094207653 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd000ba80 con 0x7fede8158fb0 2023-10-07T23:21:29.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7fede6ffd700 1 -- 172.21.15.57:0/2094207653 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00092e0 con 0x7fede8158fb0 2023-10-07T23:21:29.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- 172.21.15.57:0/2094207653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158fb0 msgr2=0x7fede80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158fb0 0x7fede80ce6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009f20 tx=0x7fedd0009840 comp rx=0 tx=0).stop 2023-10-07T23:21:29.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- 172.21.15.57:0/2094207653 shutdown_connections 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158fb0 0x7fede80ce6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 --2- 172.21.15.57:0/2094207653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158be0 0x7fede80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- 172.21.15.57:0/2094207653 >> 172.21.15.57:0/2094207653 conn(0x7fede8068520 msgr2=0x7fede8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- 172.21.15.57:0/2094207653 shutdown_connections 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.762+0000 7feded0c2700 1 -- 172.21.15.57:0/2094207653 wait complete. 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 Processor -- start 2023-10-07T23:21:29.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- start start 2023-10-07T23:21:29.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fede806b850 0x7fede814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158fb0 0x7fede80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede814f2a0 con 0x7fede8158be0 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede814dec0 con 0x7fede806b850 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede815b4b0 con 0x7fede8158fb0 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58256/0 (socket says 172.21.15.57:58256) 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 -- 172.21.15.57:0/2408517550 learned_addr learned my addr 172.21.15.57:0/2408517550 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede7fff700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fede806b850 0x7fede814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fedec8c1700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158fb0 0x7fede80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 -- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158fb0 msgr2=0x7fede80c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158fb0 0x7fede80c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 -- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fede806b850 msgr2=0x7fede814ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fede806b850 0x7fede814ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 -- 172.21.15.57:0/2408517550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009ae0 con 0x7fede8158be0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7fede77fe700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b6c0 tx=0x7fedd0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00094e0 con 0x7fede8158be0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd00078a0 con 0x7fede8158be0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede815bd50 con 0x7fede8158be0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0007c40 con 0x7fede8158be0 2023-10-07T23:21:29.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.766+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede815b700 con 0x7fede8158be0 2023-10-07T23:21:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede80b4c80 con 0x7fede8158be0 2023-10-07T23:21:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fedd0003040 con 0x7fede8158be0 2023-10-07T23:21:29.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7feddffff700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 0x7fedc8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:29.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fedd007e7e0 con 0x7fede8158be0 2023-10-07T23:21:29.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7fede7fff700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 0x7fedc8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:29.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.770+0000 7fede7fff700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 0x7fedc8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedd8006fd0 tx=0x7fedd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:29.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.778+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fede80b4c80 con 0x7fede8158be0 2023-10-07T23:21:29.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.934+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fede80b4c80 con 0x7fede8158be0 2023-10-07T23:21:29.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.934+0000 7feddffff700 1 -- 172.21.15.57:0/2408517550 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fede80b4c80 con 0x7fede8158be0 2023-10-07T23:21:29.942 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:29.942 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:29.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 msgr2=0x7fedc8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 0x7fedc8062a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedd8006fd0 tx=0x7fedd8008040 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 msgr2=0x7fede814d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b6c0 tx=0x7fedd0018000 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 shutdown_connections 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fede8158fb0 0x7fede80c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fede8158be0 0x7fede814d980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fedc8060550 0x7fedc8062a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 --2- 172.21.15.57:0/2408517550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fede806b850 0x7fede814ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 >> 172.21.15.57:0/2408517550 conn(0x7fede8068520 msgr2=0x7fede80688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 shutdown_connections 2023-10-07T23:21:29.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:29.938+0000 7feded0c2700 1 -- 172.21.15.57:0/2408517550 wait complete. 2023-10-07T23:21:29.960 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-07T23:21:29.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:30.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 Processor -- start 2023-10-07T23:21:30.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 -- start start 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa09806cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0980cb020 0x7fa0980cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0980cc4e0 con 0x7fa098156690 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0980cb470 con 0x7fa0980cb020 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa09ca1d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980cc710 con 0x7fa09806c9a0 2023-10-07T23:21:30.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa097fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa09806cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa097fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58270/0 (socket says 172.21.15.57:58270) 2023-10-07T23:21:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa096ffd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0980cb020 0x7fa0980cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa097fff700 1 -- 172.21.15.57:0/3860627064 learned_addr learned my addr 172.21.15.57:0/3860627064 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 -- 172.21.15.57:0/3860627064 >> v2:172.21.15.154:3300/0 conn(0x7fa0980cb020 msgr2=0x7fa0980cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 --2- 172.21.15.57:0/3860627064 >> v2:172.21.15.154:3300/0 conn(0x7fa0980cb020 0x7fa0980cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 -- 172.21.15.57:0/3860627064 >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 msgr2=0x7fa098156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 --2- 172.21.15.57:0/3860627064 >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 -- 172.21.15.57:0/3860627064 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980cc9f0 con 0x7fa09806c9a0 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa097fff700 1 --2- 172.21.15.57:0/3860627064 >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0977fe700 1 --2- 172.21.15.57:0/3860627064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa09806cd70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa09806dfd0 tx=0x7fa090009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc806876b2e6ae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0967fc700 1 -- 172.21.15.57:0/3860627064 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000e070 con 0x7fa09806c9a0 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0967fc700 1 -- 172.21.15.57:0/3860627064 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000bab0 con 0x7fa09806c9a0 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.062+0000 7fa0967fc700 1 -- 172.21.15.57:0/3860627064 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000be70 con 0x7fa09806c9a0 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3860627064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 msgr2=0x7fa09806cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3860627064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa09806cd70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa09806dfd0 tx=0x7fa090009780 comp rx=0 tx=0).stop 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3860627064 shutdown_connections 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3860627064 >> v2:172.21.15.57:3300/0 conn(0x7fa098156690 0x7fa098156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3860627064 >> v2:172.21.15.154:3300/0 conn(0x7fa0980cb020 0x7fa0980cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3860627064 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa09806cd70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3860627064 >> 172.21.15.57:0/3860627064 conn(0x7fa0980692f0 msgr2=0x7fa0980696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3860627064 shutdown_connections 2023-10-07T23:21:30.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3860627064 wait complete. 2023-10-07T23:21:30.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 Processor -- start 2023-10-07T23:21:30.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- start start 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa098154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 0x7fa0980acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980ad4a0 con 0x7fa098156690 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098157420 con 0x7fa0980cb020 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa09ca1d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0981597e0 con 0x7fa09806c9a0 2023-10-07T23:21:30.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa097fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa096ffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 0x7fa0980acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa0977fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa098154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa097fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58286/0 (socket says 172.21.15.57:58286) 2023-10-07T23:21:30.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.066+0000 7fa097fff700 1 -- 172.21.15.57:0/3175913590 learned_addr learned my addr 172.21.15.57:0/3175913590 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 -- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 msgr2=0x7fa098154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa098154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 -- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 msgr2=0x7fa0980acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 0x7fa0980acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 -- 172.21.15.57:0/3175913590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880094f0 con 0x7fa098156690 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa096ffd700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 0x7fa0980acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa097fff700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa08800ea40 tx=0x7fa088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08800ee80 con 0x7fa098156690 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08800c850 con 0x7fa098156690 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa090009a20 con 0x7fa098156690 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08800cbf0 con 0x7fa098156690 2023-10-07T23:21:30.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa098156dd0 con 0x7fa098156690 2023-10-07T23:21:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.070+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0980711c0 con 0x7fa098156690 2023-10-07T23:21:30.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.074+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa08800cd50 con 0x7fa098156690 2023-10-07T23:21:30.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.074+0000 7fa08f7fe700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 0x7fa078062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.074+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa08807c910 con 0x7fa098156690 2023-10-07T23:21:30.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.074+0000 7fa0977fe700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 0x7fa078062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.074+0000 7fa0977fe700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 0x7fa078062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0900097c0 tx=0x7fa090019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.078+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa088050990 con 0x7fa098156690 2023-10-07T23:21:30.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.242+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0980cc1e0 con 0x7fa098156690 2023-10-07T23:21:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.242+0000 7fa08f7fe700 1 -- 172.21.15.57:0/3175913590 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fa088050520 con 0x7fa098156690 2023-10-07T23:21:30.249 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:30.249 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 msgr2=0x7fa078062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 0x7fa078062a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0900097c0 tx=0x7fa090019000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 msgr2=0x7fa098168f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa08800ea40 tx=0x7fa088009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 shutdown_connections 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa098156690 0x7fa098168f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa078060550 0x7fa078062a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0980cb020 0x7fa0980acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 --2- 172.21.15.57:0/3175913590 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa09806c9a0 0x7fa098154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 >> 172.21.15.57:0/3175913590 conn(0x7fa0980692f0 msgr2=0x7fa0980c2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 shutdown_connections 2023-10-07T23:21:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.246+0000 7fa09ca1d700 1 -- 172.21.15.57:0/3175913590 wait complete. 2023-10-07T23:21:30.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-07T23:21:30.264 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-07T23:21:30.265 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-07T23:21:30.265 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-07T23:21:30.266 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-07T23:21:30.266 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-07T23:21:30.267 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-07T23:21:30.267 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-07T23:21:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.402+0000 7f4a1b59e700 1 Processor -- start 2023-10-07T23:21:30.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 -- start start 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4a1c06b990 0x7f4a1c0cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cc100 con 0x7f4a1c15cfe0 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c06a980 con 0x7f4a1c06b990 2023-10-07T23:21:30.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c06b500 con 0x7f4a1c15d3b0 2023-10-07T23:21:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.402+0000 7fa36e228700 1 Processor -- start 2023-10-07T23:21:30.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 -- start start 2023-10-07T23:21:30.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58296/0 (socket says 172.21.15.57:58296) 2023-10-07T23:21:30.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 -- 172.21.15.57:0/2196248483 learned_addr learned my addr 172.21.15.57:0/2196248483 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a19d9b700 1 --2- 172.21.15.57:0/2196248483 >> v2:172.21.15.154:3300/0 conn(0x7f4a1c06b990 0x7f4a1c0cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1ad9d700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 Processor -- start 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 -- start start 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 -- 172.21.15.57:0/2196248483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 msgr2=0x7f4a1c0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c0c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 -- 172.21.15.57:0/2196248483 >> v2:172.21.15.154:3300/0 conn(0x7f4a1c06b990 msgr2=0x7f4a1c0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/2196248483 >> v2:172.21.15.154:3300/0 conn(0x7f4a1c06b990 0x7f4a1c0cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 -- 172.21.15.57:0/2196248483 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c06abd0 con 0x7f4a1c15cfe0 2023-10-07T23:21:30.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0cca00 tx=0x7f4a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd71a53b2a7ff23 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1959a700 1 -- 172.21.15.57:0/2196248483 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000b720 con 0x7f4a1c15cfe0 2023-10-07T23:21:30.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1959a700 1 -- 172.21.15.57:0/2196248483 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000b880 con 0x7f4a1c15cfe0 2023-10-07T23:21:30.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4a1959a700 1 -- 172.21.15.57:0/2196248483 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000bc20 con 0x7f4a1c15cfe0 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa36806d4b0 0x7fa3680c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3680b0ec0 0x7fa3680b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3680c7540 con 0x7fa3680c96f0 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3680cc320 con 0x7fa3680b0ec0 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36e228700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3680cc040 con 0x7fa36806d4b0 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3680b0ec0 0x7fa3680b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3680b0ec0 0x7fa3680b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58312/0 (socket says 172.21.15.57:58312) 2023-10-07T23:21:30.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 -- 172.21.15.57:0/3993980520 learned_addr learned my addr 172.21.15.57:0/3993980520 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f73d7362700 1 Processor -- start 2023-10-07T23:21:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36ca25700 1 --2- 172.21.15.57:0/3993980520 >> v2:172.21.15.154:3300/0 conn(0x7fa36806d4b0 0x7fa3680c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f73d7362700 1 -- start start 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 -- 172.21.15.57:0/3993980520 >> v2:172.21.15.154:3300/0 conn(0x7fa36806d4b0 msgr2=0x7fa3680c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 --2- 172.21.15.57:0/3993980520 >> v2:172.21.15.154:3300/0 conn(0x7fa36806d4b0 0x7fa3680c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 -- 172.21.15.57:0/3993980520 >> v1:172.21.15.57:6789/0 conn(0x7fa3680c96f0 legacy=0x7fa3680cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 -- 172.21.15.57:0/3993980520 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680c7770 con 0x7fa3680b0ec0 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f48181568e0 0x7f4818156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c8ed0 0x7f481806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48180cb850 0x7f48180cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481806bc80 con 0x7f48181568e0 2023-10-07T23:21:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180cc210 con 0x7f48180cb850 2023-10-07T23:21:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7f4817fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481806b7f0 con 0x7f48180c8ed0 2023-10-07T23:21:30.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.406+0000 7fa36da27700 1 --2- 172.21.15.57:0/3993980520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3680b0ec0 0x7fa3680b1360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3680c6da0 tx=0x7fa360009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a742bddd8075f349 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- 172.21.15.57:0/2196248483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c15cfe0 msgr2=0x7f4a1c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0cca00 tx=0x7f4a10009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- 172.21.15.57:0/2196248483 shutdown_connections 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/2196248483 >> v2:172.21.15.154:3300/0 conn(0x7f4a1c06b990 0x7f4a1c0cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/2196248483 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c15cfe0 0x7f4a1c0cb680 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.420 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- 172.21.15.57:0/2196248483 >> 172.21.15.57:0/2196248483 conn(0x7f4a1c068520 msgr2=0x7f4a1c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48180cb850 0x7f48180cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48167fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c8ed0 0x7f481806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48180cb850 0x7f48180cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36894/0 (socket says 172.21.15.57:36894) 2023-10-07T23:21:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 -- 172.21.15.57:0/4221965817 learned_addr learned my addr 172.21.15.57:0/4221965817 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4816ffd700 1 --2- 172.21.15.57:0/4221965817 >> v2:172.21.15.57:3300/0 conn(0x7f48181568e0 0x7f4818156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa367fff700 1 -- 172.21.15.57:0/3993980520 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000e070 con 0x7fa3680b0ec0 2023-10-07T23:21:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa367fff700 1 -- 172.21.15.57:0/3993980520 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000ba50 con 0x7fa3680b0ec0 2023-10-07T23:21:30.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa367fff700 1 -- 172.21.15.57:0/3993980520 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600092e0 con 0x7fa3680b0ec0 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 Processor -- start 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 -- start start 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 -- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c8ed0 msgr2=0x7f481806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 --2- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c8ed0 0x7f481806a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 -- 172.21.15.57:0/4221965817 >> v2:172.21.15.57:3300/0 conn(0x7f48181568e0 msgr2=0x7f4818156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 --2- 172.21.15.57:0/4221965817 >> v2:172.21.15.57:3300/0 conn(0x7f48181568e0 0x7f4818156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 -- 172.21.15.57:0/4221965817 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481806abe0 con 0x7f48180cb850 2023-10-07T23:21:30.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- 172.21.15.57:0/2196248483 shutdown_connections 2023-10-07T23:21:30.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- 172.21.15.57:0/2196248483 wait complete. 2023-10-07T23:21:30.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f48177fe700 1 --2- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180cb850 0x7f48180cbcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48180c2920 tx=0x7f4808009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2724e03faf69fc4e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d7362700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73d0152f30 0x7f73d00afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d7362700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d006ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d7362700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d01567a0 con 0x7f73d006b380 2023-10-07T23:21:30.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d7362700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d006b820 con 0x7f73d0152f30 2023-10-07T23:21:30.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d7362700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d00af290 con 0x7f73d00b02b0 2023-10-07T23:21:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 -- 172.21.15.57:0/3993980520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3680b0ec0 msgr2=0x7fa3680b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 --2- 172.21.15.57:0/3993980520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3680b0ec0 0x7fa3680b1360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3680c6da0 tx=0x7fa360009840 comp rx=0 tx=0).stop 2023-10-07T23:21:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4815ffb700 1 -- 172.21.15.57:0/4221965817 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800b640 con 0x7f48180cb850 2023-10-07T23:21:30.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4815ffb700 1 -- 172.21.15.57:0/4221965817 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480800b7a0 con 0x7f48180cb850 2023-10-07T23:21:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4815ffb700 1 -- 172.21.15.57:0/4221965817 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800bb60 con 0x7f48180cb850 2023-10-07T23:21:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d5b5f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d006ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73d0152f30 0x7f73d00afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73d0152f30 0x7f73d00afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58322/0 (socket says 172.21.15.57:58322) 2023-10-07T23:21:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 -- 172.21.15.57:0/3246670566 learned_addr learned my addr 172.21.15.57:0/3246670566 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 -- 172.21.15.57:0/3993980520 shutdown_connections 2023-10-07T23:21:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 --2- 172.21.15.57:0/3993980520 >> v2:172.21.15.154:3300/0 conn(0x7fa36806d4b0 0x7fa3680c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 --2- 172.21.15.57:0/3993980520 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3680b0ec0 0x7fa3680b1360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 -- 172.21.15.57:0/3993980520 >> 172.21.15.57:0/3993980520 conn(0x7fa368068520 msgr2=0x7fa368068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f84156a80 0x7f0f8415b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.429 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 0x7f0f840adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 -- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180cb850 msgr2=0x7f48180cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 --2- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180cb850 0x7f48180cbcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48180c2920 tx=0x7f4808009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f840c7130 con 0x7f0f840ab740 2023-10-07T23:21:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f840c6840 con 0x7f0f84156a80 2023-10-07T23:21:30.430 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f8a7dd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840c6ae0 con 0x7f0f84156e50 2023-10-07T23:21:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 -- 172.21.15.57:0/3246670566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 msgr2=0x7f73d006ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d006ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f84156a80 0x7f0f8415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 -- 172.21.15.57:0/3246670566 >> v1:172.21.15.154:6789/0 conn(0x7f73d006b380 legacy=0x7f73d006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f84156a80 0x7f0f8415b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58338/0 (socket says 172.21.15.57:58338) 2023-10-07T23:21:30.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 -- 172.21.15.57:0/542926186 learned_addr learned my addr 172.21.15.57:0/542926186 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 -- 172.21.15.57:0/3246670566 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d00af570 con 0x7f73d0152f30 2023-10-07T23:21:30.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 -- 172.21.15.57:0/4221965817 shutdown_connections 2023-10-07T23:21:30.433 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 --2- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48180cb850 0x7f48180cbcd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 --2- 172.21.15.57:0/4221965817 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180c8ed0 0x7f481806a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 --2- 172.21.15.57:0/4221965817 >> v2:172.21.15.57:3300/0 conn(0x7f48181568e0 0x7f4818156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 -- 172.21.15.57:0/4221965817 >> 172.21.15.57:0/4221965817 conn(0x7f4818068520 msgr2=0x7f4818068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 Processor -- start 2023-10-07T23:21:30.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4a1b59e700 1 -- start start 2023-10-07T23:21:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d5b5f700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d006ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f837fe700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 0x7f0f840adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 -- 172.21.15.57:0/542926186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 msgr2=0x7f0f840adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.435 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 0x7f0f840adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 -- 172.21.15.57:0/542926186 >> v1:172.21.15.154:6789/0 conn(0x7f0f840ab740 legacy=0x7f0f840cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 -- 172.21.15.57:0/542926186 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f840c6dc0 con 0x7f0f84156a80 2023-10-07T23:21:30.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 -- 172.21.15.57:0/4221965817 shutdown_connections 2023-10-07T23:21:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f73d6360700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00afd70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d1a0 tx=0x7f73cc00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad28e17df9f31e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f837fe700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 0x7f0f840adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:30.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f83fff700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156a80 0x7f0f8415b330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009f20 tx=0x7f0f7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da45df4e597b2893 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 -- 172.21.15.57:0/3993980520 shutdown_connections 2023-10-07T23:21:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7fa36e228700 1 -- 172.21.15.57:0/3993980520 wait complete. 2023-10-07T23:21:30.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 0x7f4a1c171150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a1c0b0d00 0x7f4a1c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c171760 con 0x7f4a1c06b990 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c152370 con 0x7f4a1c0b0d00 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b0870 con 0x7f4a1c15d3b0 2023-10-07T23:21:30.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 0x7f4a1c171150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 0x7f4a1c171150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58342/0 (socket says 172.21.15.57:58342) 2023-10-07T23:21:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 -- 172.21.15.57:0/753234170 learned_addr learned my addr 172.21.15.57:0/753234170 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f4817fff700 1 -- 172.21.15.57:0/4221965817 wait complete. 2023-10-07T23:21:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f82ffd700 1 -- 172.21.15.57:0/542926186 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00e070 con 0x7f0f84156a80 2023-10-07T23:21:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f82ffd700 1 -- 172.21.15.57:0/542926186 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c00ba40 con 0x7f0f84156a80 2023-10-07T23:21:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.410+0000 7f0f82ffd700 1 -- 172.21.15.57:0/542926186 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0092e0 con 0x7f0f84156a80 2023-10-07T23:21:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/542926186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156a80 msgr2=0x7f0f8415b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156a80 0x7f0f8415b330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009f20 tx=0x7f0f7c009840 comp rx=0 tx=0).stop 2023-10-07T23:21:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1ad9d700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a1c0b0d00 0x7f4a1c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a19d9b700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/542926186 shutdown_connections 2023-10-07T23:21:30.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f84156e50 0x7f0f840adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 --2- 172.21.15.57:0/542926186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156a80 0x7f0f8415b330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/542926186 >> 172.21.15.57:0/542926186 conn(0x7f0f840670e0 msgr2=0x7f0f840674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 Processor -- start 2023-10-07T23:21:30.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/542926186 shutdown_connections 2023-10-07T23:21:30.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 -- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 msgr2=0x7f4a1c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4a1c15d3b0 0x7f4a1c15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 -- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a1c0b0d00 msgr2=0x7f4a1c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a1c0b0d00 0x7f4a1c1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100094f0 con 0x7f4a1c06b990 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/542926186 wait complete. 2023-10-07T23:21:30.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d535e700 1 -- 172.21.15.57:0/3246670566 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc010070 con 0x7f73d0152f30 2023-10-07T23:21:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 -- start start 2023-10-07T23:21:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d535e700 1 -- 172.21.15.57:0/3246670566 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00dcb0 con 0x7f73d0152f30 2023-10-07T23:21:30.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d535e700 1 -- 172.21.15.57:0/3246670566 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc016650 con 0x7f73d0152f30 2023-10-07T23:21:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1a59c700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 0x7f4a1c171150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ec20 tx=0x7f4a10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 Processor -- start 2023-10-07T23:21:30.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 Processor -- start 2023-10-07T23:21:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 Processor -- start 2023-10-07T23:21:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 -- start start 2023-10-07T23:21:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 -- 172.21.15.57:0/3246670566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 msgr2=0x7f73d00afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00afd70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00d1a0 tx=0x7f73cc00ba50 comp rx=0 tx=0).stop 2023-10-07T23:21:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 -- start start 2023-10-07T23:21:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000bc00 con 0x7f4a1c06b990 2023-10-07T23:21:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a10011420 con 0x7f4a1c06b990 2023-10-07T23:21:30.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a100117c0 con 0x7f4a1c06b990 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 0x7f48181576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180cb850 0x7f48180c1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48181568e0 0x7f48180c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481813aa70 con 0x7f48180c8ed0 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48181399d0 con 0x7f48181568e0 2023-10-07T23:21:30.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4817fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4818139cb0 con 0x7f48180cb850 2023-10-07T23:21:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa36806d4b0 0x7fa368176ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3680b0ec0 0x7fa368156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3680c96f0 0x7fa3680c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36815d020 con 0x7fa36806d4b0 2023-10-07T23:21:30.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681572b0 con 0x7fa3680c96f0 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36e228700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36815c6f0 con 0x7fa3680b0ec0 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 0x7f743406ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7434152f10 0x7f7434153370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743415d600 con 0x7f743406ca30 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74340c7100 con 0x7f7434152f10 2023-10-07T23:21:30.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f7439fd9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74341538b0 con 0x7f743415a180 2023-10-07T23:21:30.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3680c96f0 0x7fa3680c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36ca25700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3680b0ec0 0x7fa368156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36ca25700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3680b0ec0 0x7fa368156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56754/0 (socket says 172.21.15.57:56754) 2023-10-07T23:21:30.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36ca25700 1 -- 172.21.15.57:0/846416261 learned_addr learned my addr 172.21.15.57:0/846416261 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0b1170 con 0x7f4a1c06b990 2023-10-07T23:21:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0aff40 con 0x7f4a1c06b990 2023-10-07T23:21:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 0x7f48181576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 0x7f48181576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58356/0 (socket says 172.21.15.57:58356) 2023-10-07T23:21:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f48167fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180cb850 0x7f48180c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 -- 172.21.15.57:0/2076421203 learned_addr learned my addr 172.21.15.57:0/2076421203 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 -- start start 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ad806cd70 0x7f4ad8154c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad80abf70 con 0x7f4ad81551c0 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad8155620 con 0x7f4ad806c9a0 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4ade4b7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad806d1f0 con 0x7f4ad806cd70 2023-10-07T23:21:30.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 -- 172.21.15.57:0/3246670566 shutdown_connections 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d006ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 --2- 172.21.15.57:0/3246670566 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00afd70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 -- 172.21.15.57:0/3246670566 >> 172.21.15.57:0/3246670566 conn(0x7f73d00cce10 msgr2=0x7f73d00add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 -- 172.21.15.57:0/3246670566 shutdown_connections 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 Processor -- start 2023-10-07T23:21:30.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f0f8a7dd700 1 -- start start 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 -- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3680b0ec0 msgr2=0x7fa368156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3680b0ec0 0x7fa368156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 -- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa36806d4b0 msgr2=0x7fa368176ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa36806d4b0 0x7fa368176ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7fa36da27700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360009ae0 con 0x7fa3680c96f0 2023-10-07T23:21:30.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4adca54700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f4ad81551c0 0x7f4ad80aca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60458/0 (socket says 172.21.15.57:60458) 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4adca54700 1 -- 172.21.15.57:0/3214572040 learned_addr learned my addr 172.21.15.57:0/3214572040 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad77fe700 1 --2- 172.21.15.57:0/3214572040 >> v2:172.21.15.154:3300/0 conn(0x7f4ad806cd70 0x7f4ad8154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f48177fe700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48181568e0 0x7f48180c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 -- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180cb850 msgr2=0x7f48180c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f48180cb850 0x7f48180c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 -- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48181568e0 msgr2=0x7f48180c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48181568e0 0x7f48180c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f4816ffd700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080094f0 con 0x7f48180c8ed0 2023-10-07T23:21:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4816ffd700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 0x7f48181576c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f480c00ec80 tx=0x7f480c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 0x7f743406ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f840ab740 0x7f0f841516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f84156a80 0x7f0f8414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 0x7f0f84153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a1c0711c0 con 0x7f4a1c06b990 2023-10-07T23:21:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 0x7f743406ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56768/0 (socket says 172.21.15.57:56768) 2023-10-07T23:21:30.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7432ffd700 1 -- 172.21.15.57:0/2779735749 learned_addr learned my addr 172.21.15.57:0/2779735749 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 --2- 172.21.15.57:0/2779735749 >> v2:172.21.15.57:3300/0 conn(0x7f7434152f10 0x7f7434153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa36da27700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3680c96f0 0x7fa3680c4af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa360015000 tx=0x7fa360018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8414e6a0 con 0x7f0f84156e50 2023-10-07T23:21:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84150e40 con 0x7f0f84156a80 2023-10-07T23:21:30.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8414da90 con 0x7f0f840ab740 2023-10-07T23:21:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 -- 172.21.15.57:0/2779735749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 msgr2=0x7f743406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 --2- 172.21.15.57:0/2779735749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 0x7f743406ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 -- 172.21.15.57:0/2779735749 >> v1:172.21.15.57:6789/0 conn(0x7f743406ca30 legacy=0x7f74340c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad6ffd700 1 -- 172.21.15.57:0/3214572040 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008521387 0 0) 0x7f4ad8155620 con 0x7f4ad806c9a0 2023-10-07T23:21:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad6ffd700 1 -- 172.21.15.57:0/3214572040 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac0003570 con 0x7f4ad806c9a0 2023-10-07T23:21:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 -- 172.21.15.57:0/2779735749 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743415c9f0 con 0x7f7434152f10 2023-10-07T23:21:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 Processor -- start 2023-10-07T23:21:30.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 -- start start 2023-10-07T23:21:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00c640 con 0x7f48180c8ed0 2023-10-07T23:21:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad77fe700 1 -- 172.21.15.57:0/3214572040 >> v1:172.21.15.154:6789/0 conn(0x7f4ad806c9a0 legacy=0x7f4ad806af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad77fe700 1 -- 172.21.15.57:0/3214572040 >> v1:172.21.15.57:6789/0 conn(0x7f4ad81551c0 legacy=0x7f4ad80aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad77fe700 1 -- 172.21.15.57:0/3214572040 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad806b690 con 0x7f4ad806cd70 2023-10-07T23:21:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.414+0000 7f73d7362700 1 -- 172.21.15.57:0/3246670566 wait complete. 2023-10-07T23:21:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 Processor -- start 2023-10-07T23:21:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f83fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f840ab740 0x7f0f841516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 0x7f0f84153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 0x7f0f84153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58370/0 (socket says 172.21.15.57:58370) 2023-10-07T23:21:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 -- 172.21.15.57:0/262371034 learned_addr learned my addr 172.21.15.57:0/262371034 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f837fe700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f84156a80 0x7f0f8414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c00c7a0 con 0x7f48180c8ed0 2023-10-07T23:21:30.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4817fff700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4818139f90 con 0x7f48180c8ed0 2023-10-07T23:21:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600032f0 con 0x7fa3680c96f0 2023-10-07T23:21:30.467 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00cb40 con 0x7f48180c8ed0 2023-10-07T23:21:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 -- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f840ab740 msgr2=0x7f0f841516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f840ab740 0x7f0f841516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 -- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f84156a80 msgr2=0x7f0f8414cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f84156a80 0x7f0f8414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad77fe700 1 --2- 172.21.15.57:0/3214572040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806cd70 0x7f4ad8154c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000ae00 tx=0x7f4ad000a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde222711b01dfde server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009ae0 con 0x7f0f84156e50 2023-10-07T23:21:30.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7433fff700 1 --2- 172.21.15.57:0/2779735749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7434152f10 0x7f7434153370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f74240098e0 tx=0x7f74240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d289958345f8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f74327fc700 1 -- 172.21.15.57:0/2779735749 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400b640 con 0x7f7434152f10 2023-10-07T23:21:30.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36815c9d0 con 0x7fa3680c96f0 2023-10-07T23:21:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3680c5030 con 0x7fa3680c96f0 2023-10-07T23:21:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa360008dd0 con 0x7fa3680c96f0 2023-10-07T23:21:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600266b0 con 0x7fa3680c96f0 2023-10-07T23:21:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f88d7a700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 0x7f0f84153110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400c5b0 tx=0x7f0f74007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f74327fc700 1 -- 172.21.15.57:0/2779735749 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742400b7a0 con 0x7f7434152f10 2023-10-07T23:21:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad6ffd700 1 -- 172.21.15.57:0/3214572040 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000f640 con 0x7f4ad806cd70 2023-10-07T23:21:30.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 -- start start 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d006b380 0x7f73d01520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d00a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d00aa360 con 0x7f73d0152f30 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0152620 con 0x7f73d006b380 2023-10-07T23:21:30.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d7362700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d00a82b0 con 0x7f73d00b02b0 2023-10-07T23:21:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7439fd9700 1 -- 172.21.15.57:0/2779735749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7434152f10 msgr2=0x7f7434153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f7439fd9700 1 --2- 172.21.15.57:0/2779735749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7434152f10 0x7f7434153370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f74240098e0 tx=0x7f74240091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 0x7ff5140bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 0x7ff51406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51406b270 con 0x7ff51406b700 2023-10-07T23:21:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5140c6b00 con 0x7ff5140becc0 2023-10-07T23:21:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff51af1c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51406bb50 con 0x7ff5141618c0 2023-10-07T23:21:30.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4817fff700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481813a5d0 con 0x7f48180c8ed0 2023-10-07T23:21:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011590 con 0x7f0f84156e50 2023-10-07T23:21:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f74011710 con 0x7f0f84156e50 2023-10-07T23:21:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011ad0 con 0x7f0f84156e50 2023-10-07T23:21:30.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8414dd70 con 0x7f0f84156e50 2023-10-07T23:21:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8414e330 con 0x7f0f84156e50 2023-10-07T23:21:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f840711c0 con 0x7f0f84156e50 2023-10-07T23:21:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad6ffd700 1 -- 172.21.15.57:0/3214572040 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000f7c0 con 0x7f4ad806cd70 2023-10-07T23:21:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ad6ffd700 1 -- 172.21.15.57:0/3214572040 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000fb80 con 0x7f4ad806cd70 2023-10-07T23:21:30.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ade4b7700 1 -- 172.21.15.57:0/3214572040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806cd70 msgr2=0x7f4ad8154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ade4b7700 1 --2- 172.21.15.57:0/3214572040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806cd70 0x7f4ad8154c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000ae00 tx=0x7f4ad000a6a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa348000fd0 con 0x7fa3680c96f0 2023-10-07T23:21:30.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ade4b7700 1 -- 172.21.15.57:0/3214572040 shutdown_connections 2023-10-07T23:21:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ade4b7700 1 --2- 172.21.15.57:0/3214572040 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806cd70 0x7f4ad8154c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4ade4b7700 1 -- 172.21.15.57:0/3214572040 >> 172.21.15.57:0/3214572040 conn(0x7f4ad8068740 msgr2=0x7f4ad8068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4a10011b70 con 0x7f4a1c06b990 2023-10-07T23:21:30.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff513fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff51406b700 0x7ff5141611b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60478/0 (socket says 172.21.15.57:60478) 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d6b61700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d6360700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d006b380 0x7f73d01520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d6b61700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58378/0 (socket says 172.21.15.57:58378) 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d6360700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d006b380 0x7f73d01520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36926/0 (socket says 172.21.15.57:36926) 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7f73d6b61700 1 -- 172.21.15.57:0/3805131316 learned_addr learned my addr 172.21.15.57:0/3805131316 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.418+0000 7ff513fff700 1 -- 172.21.15.57:0/4136737038 learned_addr learned my addr 172.21.15.57:0/4136737038 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f7439fd9700 1 -- 172.21.15.57:0/2779735749 shutdown_connections 2023-10-07T23:21:30.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f7439fd9700 1 --2- 172.21.15.57:0/2779735749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f743415a180 0x7f743406ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f7439fd9700 1 --2- 172.21.15.57:0/2779735749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7434152f10 0x7f7434153370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f7439fd9700 1 -- 172.21.15.57:0/2779735749 >> 172.21.15.57:0/2779735749 conn(0x7f7434068520 msgr2=0x7f7434068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4817fff700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48180711c0 con 0x7f48180c8ed0 2023-10-07T23:21:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4ade4b7700 1 -- 172.21.15.57:0/3214572040 shutdown_connections 2023-10-07T23:21:30.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49fc0606a0 0x7f49fc062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a1007e410 con 0x7f4a1c06b990 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7f49fc0688b0 0x7f49fc06acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49fc06b370 con 0x7f49fc0688b0 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a1007e740 con 0x7f4a1c06b990 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a19d9b700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49fc0606a0 0x7f49fc062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a1ad9d700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7f49fc0688b0 0x7f49fc06acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5194b9700 1 --2- 172.21.15.57:0/4136737038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 0x7ff51406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff518cb8700 1 --2- 172.21.15.57:0/4136737038 >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 0x7ff5140bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a1ad9d700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7f49fc0688b0 0x7f49fc06acc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5137fe700 1 -- 172.21.15.57:0/4136737038 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777026922 0 0) 0x7ff51406b270 con 0x7ff51406b700 2023-10-07T23:21:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5137fe700 1 -- 172.21.15.57:0/4136737038 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4fc003570 con 0x7ff51406b700 2023-10-07T23:21:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa360003040 con 0x7fa3680c96f0 2023-10-07T23:21:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa354060620 0x7fa354062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa36007eeb0 con 0x7fa3680c96f0 2023-10-07T23:21:30.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4ade4b7700 1 -- 172.21.15.57:0/3214572040 wait complete. 2023-10-07T23:21:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa36d226700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa354060620 0x7fa354062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7fa354068830 0x7fa35406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa35406b2f0 con 0x7fa354068830 2023-10-07T23:21:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa36007f280 con 0x7fa3680c96f0 2023-10-07T23:21:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a19d9b700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49fc0606a0 0x7f49fc062b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c171690 tx=0x7f4a0c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4ade4b7700 1 Processor -- start 2023-10-07T23:21:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f480c013470 con 0x7f48180c8ed0 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4804060620 0x7f4804062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f480c07cbb0 con 0x7f48180c8ed0 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f4804068830 0x7f480406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f480406b2f0 con 0x7f4804068830 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480c07cf80 con 0x7f48180c8ed0 2023-10-07T23:21:30.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== osd.3 v2:172.21.15.57:6808/33394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f49fc06b370 con 0x7f49fc0688b0 2023-10-07T23:21:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa36ca25700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7fa354068830 0x7fa35406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 -- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 msgr2=0x7f73d00a7980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:21:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73d00b02b0 0x7f73d00a7980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 -- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d006b380 msgr2=0x7f73d01520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73d006b380 0x7f73d01520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc00bcf0 con 0x7f73d0152f30 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d6b61700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00caa50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73c800efd0 tx=0x7f73c8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8013070 con 0x7f73d0152f30 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c800e2e0 con 0x7f73d0152f30 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c800c6a0 con 0x7f73d0152f30 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d00a8590 con 0x7f73d0152f30 2023-10-07T23:21:30.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d00a8b50 con 0x7f73d0152f30 2023-10-07T23:21:30.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0f7401c460 con 0x7f0f84156e50 2023-10-07T23:21:30.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f70060620 0x7f0f70062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f48177fe700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f4804068830 0x7f480406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f7407ce80 con 0x7f0f84156e50 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa36ca25700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7fa354068830 0x7fa35406ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7fa36d226700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa354060620 0x7fa354062ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3680c27c0 tx=0x7fa35c007b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5194b9700 1 -- 172.21.15.57:0/4136737038 >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 msgr2=0x7ff5140bf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5194b9700 1 --2- 172.21.15.57:0/4136737038 >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 0x7ff5140bf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5194b9700 1 -- 172.21.15.57:0/4136737038 >> v1:172.21.15.57:6789/0 conn(0x7ff51406b700 legacy=0x7ff5141611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.492 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7ff5194b9700 1 -- 172.21.15.57:0/4136737038 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51406ac60 con 0x7ff5141618c0 2023-10-07T23:21:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f48167fc700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4804060620 0x7f4804062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f4ade4b7700 1 -- start start 2023-10-07T23:21:30.493 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7f0f70068820 0x7f0f7006ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f7006b2e0 con 0x7f0f70068820 2023-10-07T23:21:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f7407d250 con 0x7f0f84156e50 2023-10-07T23:21:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f7439fd9700 1 -- 172.21.15.57:0/2779735749 shutdown_connections 2023-10-07T23:21:30.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 -- 172.21.15.57:0/2779735749 wait complete. 2023-10-07T23:21:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff5194b9700 1 --2- 172.21.15.57:0/4136737038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 0x7ff51406c800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5080098e0 tx=0x7ff5080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6a81f9ed4ff1d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff5137fe700 1 -- 172.21.15.57:0/4136737038 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508011630 con 0x7ff5141618c0 2023-10-07T23:21:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff5137fe700 1 -- 172.21.15.57:0/4136737038 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508011790 con 0x7ff5141618c0 2023-10-07T23:21:30.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff5137fe700 1 -- 172.21.15.57:0/4136737038 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508011b30 con 0x7ff5141618c0 2023-10-07T23:21:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== osd.1 v2:172.21.15.57:6816/33405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa35406b2f0 con 0x7fa354068830 2023-10-07T23:21:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806c9a0 0x7f4ad814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 0x7f4ad80c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad81551c0 0x7f4ad80c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80c4810 con 0x7f4ad806cd70 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80af8d0 con 0x7f4ad806c9a0 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80cc300 con 0x7f4ad81551c0 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f83fff700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f70060620 0x7f0f70062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.422+0000 7f0f837fe700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7f0f70068820 0x7f0f7006ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f0f83fff700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f70060620 0x7f0f70062ad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009880 tx=0x7f0f7c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 0x7f4ad80c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 0x7f4ad80c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58402/0 (socket says 172.21.15.57:58402) 2023-10-07T23:21:30.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 -- 172.21.15.57:0/912402943 learned_addr learned my addr 172.21.15.57:0/912402943 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad7fff700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806c9a0 0x7f4ad814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 Processor -- start 2023-10-07T23:21:30.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 -- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad81551c0 msgr2=0x7f4ad80c32f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4ad81551c0 0x7f4ad80c32f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 -- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806c9a0 msgr2=0x7f4ad814b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad806c9a0 0x7f4ad814b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad000a940 con 0x7f4ad806cd70 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f73c800ca50 con 0x7f73d0152f30 2023-10-07T23:21:30.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73d00aa130 con 0x7f73d0152f30 2023-10-07T23:21:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f73b4060550 0x7f73b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73c8080600 con 0x7f73d0152f30 2023-10-07T23:21:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f73b40686e0 0x7f73b406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73b406b1a0 con 0x7f73b40686e0 2023-10-07T23:21:30.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c8050bf0 con 0x7f73d0152f30 2023-10-07T23:21:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ad77fe700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 0x7f4ad80c2db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0014000 tx=0x7f4ad0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73d6360700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f73b4060550 0x7f73b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 -- 172.21.15.57:0/4136737038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 msgr2=0x7ff51406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 --2- 172.21.15.57:0/4136737038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 0x7ff51406c800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff5080098e0 tx=0x7ff5080091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 -- 172.21.15.57:0/4136737038 shutdown_connections 2023-10-07T23:21:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 --2- 172.21.15.57:0/4136737038 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff5141618c0 0x7ff51406c800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 --2- 172.21.15.57:0/4136737038 >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 0x7ff5140bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 -- 172.21.15.57:0/4136737038 >> 172.21.15.57:0/4136737038 conn(0x7ff514068740 msgr2=0x7ff514068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000ced0 con 0x7f4ad806cd70 2023-10-07T23:21:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f0f837fe700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7f0f70068820 0x7f0f7006ac30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff518cb8700 1 --2- 172.21.15.57:0/4136737038 >> v2:172.21.15.57:3300/0 conn(0x7ff5140becc0 0x7ff5140bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f48177fe700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f4804068830 0x7f480406ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73d5b5f700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f73b40686e0 0x7f73b406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 -- 172.21.15.57:0/4136737038 shutdown_connections 2023-10-07T23:21:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7ff51af1c700 1 -- 172.21.15.57:0/4136737038 wait complete. 2023-10-07T23:21:30.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad000f930 con 0x7f4ad806cd70 2023-10-07T23:21:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad80cc5e0 con 0x7f4ad806cd70 2023-10-07T23:21:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad80af560 con 0x7f4ad806cd70 2023-10-07T23:21:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000f030 con 0x7f4ad806cd70 2023-10-07T23:21:30.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 -- start start 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f743406ca30 0x7f74340a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 0x7f74340a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f743415a180 0x7f74340afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340a44c0 con 0x7f743406ca30 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74340a46d0 con 0x7f743415a180 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7439fd9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74340a49b0 con 0x7f7434152f10 2023-10-07T23:21:30.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 0x7f74340a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f73d5b5f700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f73b40686e0 0x7f73b406aaf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7432ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 0x7f74340a7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56798/0 (socket says 172.21.15.57:56798) 2023-10-07T23:21:30.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7432ffd700 1 -- 172.21.15.57:0/419520463 learned_addr learned my addr 172.21.15.57:0/419520463 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f7433fff700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f743415a180 0x7f74340afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ad80647b0 con 0x7f4ad806cd70 2023-10-07T23:21:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.426+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== osd.4 v2:172.21.15.154:6808/33720 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f480406b2f0 con 0x7f4804068830 2023-10-07T23:21:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 -- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f743415a180 msgr2=0x7f74340afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f743415a180 0x7f74340afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 -- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f743406ca30 msgr2=0x7f74340a5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f743406ca30 0x7f74340a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009420 con 0x7f7434152f10 2023-10-07T23:21:30.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== osd.7 v2:172.21.15.154:6812/33722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0f7006b2e0 con 0x7f0f70068820 2023-10-07T23:21:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 Processor -- start 2023-10-07T23:21:30.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7432ffd700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 0x7f74340a7fe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f742800e950 tx=0x7f7428009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f73d6360700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f73b4060550 0x7f73b4062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00ba90 tx=0x7f73cc01c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== osd.6 v2:172.21.15.154:6800/33721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f73b406b1a0 con 0x7f73b40686e0 2023-10-07T23:21:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- start start 2023-10-07T23:21:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff51406b700 0x7ff5140b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 0x7ff5140c0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.515 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5141755e0 0x7ff5140c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51415ea70 con 0x7ff5141618c0 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5140c0750 con 0x7ff5141755e0 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5140b0e70 con 0x7ff51406b700 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff513fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 0x7ff5140c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff51406b700 0x7ff5140b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff513fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 0x7ff5140c0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58422/0 (socket says 172.21.15.57:58422) 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff513fff700 1 -- 172.21.15.57:0/359277116 learned_addr learned my addr 172.21.15.57:0/359277116 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 -- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5141755e0 msgr2=0x7ff5140c1cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5141755e0 0x7ff5140c1cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 -- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 msgr2=0x7ff5140c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 0x7ff5140c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508009420 con 0x7ff51406b700 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff513fff700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff5141618c0 0x7ff5140c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff518cb8700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff51406b700 0x7ff5140b0850 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5140cb7d0 tx=0x7ff50c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f48167fc700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4804060620 0x7f4804062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4818165c00 tx=0x7f481000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00cbf0 con 0x7ff51406b700 2023-10-07T23:21:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c00f040 con 0x7ff51406b700 2023-10-07T23:21:30.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c014480 con 0x7ff51406b700 2023-10-07T23:21:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5140b1150 con 0x7ff51406b700 2023-10-07T23:21:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5140b1710 con 0x7ff51406b700 2023-10-07T23:21:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428013680 con 0x7f7434152f10 2023-10-07T23:21:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280137e0 con 0x7f7434152f10 2023-10-07T23:21:30.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74340a4c90 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428013b80 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74340a8520 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.430+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74340711c0 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f742800c490 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7420060620 0x7f7420062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f742807cc60 con 0x7f7434152f10 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f74337fe700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7420060620 0x7f7420062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7420068830 0x7f742006ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f742006b2f0 con 0x7f7420068830 2023-10-07T23:21:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f742807d030 con 0x7f7434152f10 2023-10-07T23:21:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f7433fff700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7420068830 0x7f742006ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f74337fe700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7420060620 0x7f7420062ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f742c001d90 tx=0x7f742c007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f7433fff700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7420068830 0x7f742006ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.434+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== osd.0 v2:172.21.15.57:6824/33391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f742006b2f0 con 0x7f7420068830 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5140647b0 con 0x7ff51406b700 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff50c014870 con 0x7ff51406b700 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff4fc060980 0x7ff4fc062e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff50c07d420 con 0x7ff51406b700 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7ff4fc068b10 0x7ff4fc06af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4fc06b5d0 con 0x7ff4fc068b10 2023-10-07T23:21:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4ad0003040 con 0x7f4ad806cd70 2023-10-07T23:21:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff50c009880 con 0x7ff51406b700 2023-10-07T23:21:30.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff513fff700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff4fc060980 0x7ff4fc062e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ac4060620 0x7f4ac4062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ad007e460 con 0x7f4ad806cd70 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4ad7fff700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ac4060620 0x7f4ac4062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f4ac40686c0 0x7f4ac406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ac406b180 con 0x7f4ac40686c0 2023-10-07T23:21:30.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ad007e800 con 0x7f4ad806cd70 2023-10-07T23:21:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7f4adca54700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f4ac40686c0 0x7f4ac406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.438+0000 7ff513fff700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff4fc060980 0x7ff4fc062e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5140bf350 tx=0x7ff504009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.442+0000 7f4ad7fff700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ac4060620 0x7f4ac4062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007fd0 tx=0x7f4acc007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.442+0000 7f4adca54700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f4ac40686c0 0x7f4ac406aad0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.442+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== osd.5 v2:172.21.15.154:6824/33759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4ac406b180 con 0x7f4ac40686c0 2023-10-07T23:21:30.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4817fff700 1 -- 172.21.15.57:0/2076421203 --> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f481806aec0 con 0x7f4804068830 2023-10-07T23:21:30.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 --> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa348002c90 con 0x7fa354068830 2023-10-07T23:21:30.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 --> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a1c0711c0 con 0x7f49fc0688b0 2023-10-07T23:21:30.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7ff5194b9700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7ff4fc068b10 0x7ff4fc06af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa3657fa700 1 -- 172.21.15.57:0/846416261 <== osd.1 v2:172.21.15.57:6816/33405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa348002c90 con 0x7fa354068830 2023-10-07T23:21:30.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a0a7fc700 1 -- 172.21.15.57:0/753234170 <== osd.3 v2:172.21.15.57:6808/33394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a1c0711c0 con 0x7f49fc0688b0 2023-10-07T23:21:30.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f47feffd700 1 -- 172.21.15.57:0/2076421203 <== osd.4 v2:172.21.15.154:6808/33720 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f481806aec0 con 0x7f4804068830 2023-10-07T23:21:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7f49fc0688b0 msgr2=0x7f49fc06acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7f49fc0688b0 0x7f49fc06acc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7ff5194b9700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7ff4fc068b10 0x7ff4fc06af20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49fc0606a0 msgr2=0x7f49fc062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49fc0606a0 0x7f49fc062b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c171690 tx=0x7f4a0c008040 comp rx=0 tx=0).stop 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 msgr2=0x7f4a1c171150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 --2- 172.21.15.57:0/753234170 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4a1c06b990 0x7f4a1c171150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ec20 tx=0x7f4a10017000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1ad9d700 1 -- 172.21.15.57:0/753234170 reap_dead start 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 shutdown_connections 2023-10-07T23:21:30.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 >> 172.21.15.57:0/753234170 conn(0x7f4a1c068520 msgr2=0x7f4a1c15fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7fa354068830 msgr2=0x7fa35406ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7fa354068830 0x7fa35406ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa354060620 msgr2=0x7fa354062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa354060620 0x7fa354062ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa3680c27c0 tx=0x7fa35c007b40 comp rx=0 tx=0).stop 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3680c96f0 msgr2=0x7fa3680c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 --2- 172.21.15.57:0/846416261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3680c96f0 0x7fa3680c4af0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa360015000 tx=0x7fa360018000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36da27700 1 -- 172.21.15.57:0/846416261 reap_dead start 2023-10-07T23:21:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 shutdown_connections 2023-10-07T23:21:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 >> 172.21.15.57:0/846416261 conn(0x7fa368068520 msgr2=0x7fa3680c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 shutdown_connections 2023-10-07T23:21:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.446+0000 7f4a1b59e700 1 -- 172.21.15.57:0/753234170 wait complete. 2023-10-07T23:21:30.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f4804068830 msgr2=0x7f480406ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f4804068830 0x7f480406ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 shutdown_connections 2023-10-07T23:21:30.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7fa36e228700 1 -- 172.21.15.57:0/846416261 wait complete. 2023-10-07T23:21:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4804060620 msgr2=0x7f4804062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4804060620 0x7f4804062ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4818165c00 tx=0x7f481000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:30.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== osd.2 v2:172.21.15.57:6800/33393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff4fc06b5d0 con 0x7ff4fc068b10 2023-10-07T23:21:30.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 msgr2=0x7f48181576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 --2- 172.21.15.57:0/2076421203 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48180c8ed0 0x7f48181576c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f480c00ec80 tx=0x7f480c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f48177fe700 1 -- 172.21.15.57:0/2076421203 reap_dead start 2023-10-07T23:21:30.535 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 shutdown_connections 2023-10-07T23:21:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 >> 172.21.15.57:0/2076421203 conn(0x7f4818068520 msgr2=0x7f48180c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 shutdown_connections 2023-10-07T23:21:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.450+0000 7f47fcff9700 1 -- 172.21.15.57:0/2076421203 wait complete. 2023-10-07T23:21:30.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.458+0000 7f0f8a7dd700 1 -- 172.21.15.57:0/262371034 --> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f840ae340 con 0x7f0f70068820 2023-10-07T23:21:30.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.458+0000 7f0f6bfff700 1 -- 172.21.15.57:0/262371034 <== osd.7 v2:172.21.15.154:6812/33722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0f840ae340 con 0x7f0f70068820 2023-10-07T23:21:30.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 --> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f743415ccd0 con 0x7f7420068830 2023-10-07T23:21:30.538 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.539 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-07T23:21:30.539 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 --> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f73d00aa130 con 0x7f73b40686e0 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7f0f70068820 msgr2=0x7f0f7006ac30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7f0f70068820 0x7f0f7006ac30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f70060620 msgr2=0x7f0f70062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f70060620 0x7f0f70062ad0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009880 tx=0x7f0f7c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 msgr2=0x7f0f84153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 --2- 172.21.15.57:0/262371034 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f84156e50 0x7f0f84153110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400c5b0 tx=0x7f0f74007f60 comp rx=0 tx=0).stop 2023-10-07T23:21:30.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f88d7a700 1 -- 172.21.15.57:0/262371034 reap_dead start 2023-10-07T23:21:30.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 shutdown_connections 2023-10-07T23:21:30.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 >> 172.21.15.57:0/262371034 conn(0x7f0f840670e0 msgr2=0x7f0f8414f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 shutdown_connections 2023-10-07T23:21:30.542 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f0f69ffb700 1 -- 172.21.15.57:0/262371034 wait complete. 2023-10-07T23:21:30.543 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.462+0000 7f741b7fe700 1 -- 172.21.15.57:0/419520463 <== osd.0 v2:172.21.15.57:6824/33391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f743415ccd0 con 0x7f7420068830 2023-10-07T23:21:30.545 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73c67fc700 1 -- 172.21.15.57:0/3805131316 <== osd.6 v2:172.21.15.154:6800/33721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f73d00aa130 con 0x7f73b40686e0 2023-10-07T23:21:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7420068830 msgr2=0x7f742006ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7420068830 0x7f742006ac40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7420060620 msgr2=0x7f7420062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7420060620 0x7f7420062ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f742c001d90 tx=0x7f742c007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 msgr2=0x7f74340a7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 --2- 172.21.15.57:0/419520463 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7434152f10 0x7f74340a7fe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f742800e950 tx=0x7f7428009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7433fff700 1 -- 172.21.15.57:0/419520463 reap_dead start 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 shutdown_connections 2023-10-07T23:21:30.547 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 >> 172.21.15.57:0/419520463 conn(0x7f7434068520 msgr2=0x7f74340c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.548 INFO:teuthology.orchestra.run.smithi057.stdout:60129542151 2023-10-07T23:21:30.548 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-07T23:21:30.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-07T23:21:30.550 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 shutdown_connections 2023-10-07T23:21:30.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f7439fd9700 1 -- 172.21.15.57:0/419520463 wait complete. 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f73b40686e0 msgr2=0x7f73b406aaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f73b40686e0 0x7f73b406aaf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f73b4060550 msgr2=0x7f73b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f73b4060550 0x7f73b4062a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00ba90 tx=0x7f73cc01c000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 msgr2=0x7f73d00caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 --2- 172.21.15.57:0/3805131316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73d0152f30 0x7f73d00caa50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f73c800efd0 tx=0x7f73c8007f40 comp rx=0 tx=0).stop 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d6b61700 1 -- 172.21.15.57:0/3805131316 reap_dead start 2023-10-07T23:21:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 shutdown_connections 2023-10-07T23:21:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 >> 172.21.15.57:0/3805131316 conn(0x7f73d00cce10 msgr2=0x7f73d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 shutdown_connections 2023-10-07T23:21:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f73d7362700 1 -- 172.21.15.57:0/3805131316 wait complete. 2023-10-07T23:21:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 --> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ad806b950 con 0x7f4ac40686c0 2023-10-07T23:21:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f4abffff700 1 -- 172.21.15.57:0/912402943 <== osd.5 v2:172.21.15.154:6824/33759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ad806b950 con 0x7f4ac40686c0 2023-10-07T23:21:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f4ac40686c0 msgr2=0x7f4ac406aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.466+0000 7f4ade4b7700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f4ac40686c0 0x7f4ac406aad0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ac4060620 msgr2=0x7f4ac4062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ac4060620 0x7f4ac4062ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007fd0 tx=0x7f4acc007d50 comp rx=0 tx=0).stop 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 msgr2=0x7f4ad80c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 --2- 172.21.15.57:0/912402943 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ad806cd70 0x7f4ad80c2db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0014000 tx=0x7f4ad0018000 comp rx=0 tx=0).stop 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4adca54700 1 -- 172.21.15.57:0/912402943 reap_dead start 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 shutdown_connections 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 >> 172.21.15.57:0/912402943 conn(0x7f4ad8068740 msgr2=0x7f4ad80c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 shutdown_connections 2023-10-07T23:21:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.470+0000 7f4ade4b7700 1 -- 172.21.15.57:0/912402943 wait complete. 2023-10-07T23:21:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 --> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff514161d70 con 0x7ff4fc068b10 2023-10-07T23:21:30.556 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.557 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-07T23:21:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff510ff9700 1 -- 172.21.15.57:0/359277116 <== osd.2 v2:172.21.15.57:6800/33393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff514161d70 con 0x7ff4fc068b10 2023-10-07T23:21:30.558 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-07T23:21:30.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7ff4fc068b10 msgr2=0x7ff4fc06af20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7ff4fc068b10 0x7ff4fc06af20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff4fc060980 msgr2=0x7ff4fc062e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff4fc060980 0x7ff4fc062e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff5140bf350 tx=0x7ff504009040 comp rx=0 tx=0).stop 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff51406b700 msgr2=0x7ff5140b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 --2- 172.21.15.57:0/359277116 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff51406b700 0x7ff5140b0850 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5140cb7d0 tx=0x7ff50c00b040 comp rx=0 tx=0).stop 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff5194b9700 1 -- 172.21.15.57:0/359277116 reap_dead start 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 shutdown_connections 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 >> 172.21.15.57:0/359277116 conn(0x7ff514068740 msgr2=0x7ff5140c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 shutdown_connections 2023-10-07T23:21:30.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.478+0000 7ff51af1c700 1 -- 172.21.15.57:0/359277116 wait complete. 2023-10-07T23:21:30.561 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.561 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-07T23:21:30.562 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.563 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-07T23:21:30.564 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:30.564 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-07T23:21:30.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 Processor -- start 2023-10-07T23:21:30.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 -- start start 2023-10-07T23:21:30.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 Processor -- start 2023-10-07T23:21:30.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 -- start start 2023-10-07T23:21:30.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6b240ad3e0 0x7f6b2406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b2406c800 0x7f6b24154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2406b320 con 0x7f6b2406ce80 2023-10-07T23:21:30.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2406b100 con 0x7f6b240ad3e0 2023-10-07T23:21:30.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b2b8b3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2406b600 con 0x7f6b2406c800 2023-10-07T23:21:30.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f5c2603a700 1 Processor -- start 2023-10-07T23:21:30.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f5c2603a700 1 -- start start 2023-10-07T23:21:30.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7fbddd196700 1 Processor -- start 2023-10-07T23:21:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7fbddd196700 1 -- start start 2023-10-07T23:21:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b2406c800 0x7f6b24154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b2406c800 0x7f6b24154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36950/0 (socket says 172.21.15.57:36950) 2023-10-07T23:21:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 -- 172.21.15.57:0/1787479489 learned_addr learned my addr 172.21.15.57:0/1787479489 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b28e4e700 1 --2- 172.21.15.57:0/1787479489 >> v2:172.21.15.57:3300/0 conn(0x7f6b240ad3e0 0x7f6b2406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 0x7f7a5c163d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0cb040 con 0x7f7a5c06cf10 2023-10-07T23:21:30.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5c0ca4a0 con 0x7f7a5c158be0 2023-10-07T23:21:30.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f7a62ab9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c15d9f0 con 0x7f7a5c0af530 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 -- 172.21.15.57:0/1787479489 >> v2:172.21.15.57:3300/0 conn(0x7f6b240ad3e0 msgr2=0x7f6b2406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 --2- 172.21.15.57:0/1787479489 >> v2:172.21.15.57:3300/0 conn(0x7f6b240ad3e0 0x7f6b2406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 -- 172.21.15.57:0/1787479489 >> v1:172.21.15.154:6789/0 conn(0x7f6b2406ce80 legacy=0x7f6b2406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.742+0000 7f6b29e50700 1 -- 172.21.15.57:0/1787479489 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2406b8e0 con 0x7f6b2406c800 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5bfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7a5c06cf10 0x7f7a5c0aee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60480/0 (socket says 172.21.15.57:60480) 2023-10-07T23:21:30.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5bfff700 1 -- 172.21.15.57:0/786278907 learned_addr learned my addr 172.21.15.57:0/786278907 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b29e50700 1 --2- 172.21.15.57:0/1787479489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406c800 0x7f6b24154830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b240c7b30 tx=0x7f6b20009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dbaf2ba024120fa server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c200c8500 0x7f5c200c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 0x7f5c2015d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200ca9a0 con 0x7f5c200cae30 2023-10-07T23:21:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbddd196700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbddd196700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdd806d440 0x7fbdd80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200c9ab0 con 0x7f5c200c8500 2023-10-07T23:21:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c9d90 con 0x7f5c2015ceb0 2023-10-07T23:21:30.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbddd196700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80c7170 con 0x7fbdd81554a0 2023-10-07T23:21:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a61056700 1 --2- 172.21.15.57:0/786278907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 0x7f7a5c163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbddd196700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd806b940 con 0x7fbdd806d440 2023-10-07T23:21:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbddd196700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd806ae40 con 0x7fbdd81550d0 2023-10-07T23:21:30.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5b7fe700 1 -- 172.21.15.57:0/786278907 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626787381 0 0) 0x7f7a5c0cb040 con 0x7f7a5c06cf10 2023-10-07T23:21:30.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5b7fe700 1 -- 172.21.15.57:0/786278907 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a44003570 con 0x7f7a5c06cf10 2023-10-07T23:21:30.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdd806d440 0x7fbdd80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdd806d440 0x7fbdd80cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58454/0 (socket says 172.21.15.57:58454) 2023-10-07T23:21:30.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 -- 172.21.15.57:0/3676775939 learned_addr learned my addr 172.21.15.57:0/3676775939 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 Processor -- start 2023-10-07T23:21:30.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58446/0 (socket says 172.21.15.57:58446) 2023-10-07T23:21:30.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1effd700 1 -- 172.21.15.57:0/1882689892 learned_addr learned my addr 172.21.15.57:0/1882689892 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/1882689892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 0x7f5c2015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd7fff700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 -- 172.21.15.57:0/3676775939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 msgr2=0x7fbdd80ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 -- 172.21.15.57:0/3676775939 >> v1:172.21.15.154:6789/0 conn(0x7fbdd81554a0 legacy=0x7fbdd806a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 -- 172.21.15.57:0/3676775939 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80c6e40 con 0x7fbdd806d440 2023-10-07T23:21:30.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 -- start start 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b1bfff700 1 -- 172.21.15.57:0/1787479489 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20009e70 con 0x7f6b2406c800 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a61056700 1 -- 172.21.15.57:0/786278907 >> v1:172.21.15.154:6789/0 conn(0x7f7a5c158be0 legacy=0x7f7a5c06c800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 -- 172.21.15.57:0/1882689892 >> v2:172.21.15.154:3300/0 conn(0x7f5c200c8500 msgr2=0x7f5c200c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/1882689892 >> v2:172.21.15.154:3300/0 conn(0x7f5c200c8500 0x7f5c200c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 -- 172.21.15.57:0/1882689892 >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 msgr2=0x7f5c200c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/1882689892 >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 -- 172.21.15.57:0/1882689892 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200ca070 con 0x7f5c2015ceb0 2023-10-07T23:21:30.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a61056700 1 -- 172.21.15.57:0/786278907 >> v1:172.21.15.57:6789/0 conn(0x7f7a5c06cf10 legacy=0x7f7a5c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a61056700 1 -- 172.21.15.57:0/786278907 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0ca710 con 0x7f7a5c0af530 2023-10-07T23:21:30.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1effd700 1 --2- 172.21.15.57:0/1882689892 >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd77fe700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd806d440 0x7fbdd80cbda0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00bc50 tx=0x7fbdcc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf648e1727f323d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b1bfff700 1 -- 172.21.15.57:0/1787479489 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000b860 con 0x7f6b2406c800 2023-10-07T23:21:30.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/1882689892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 0x7f5c2015d330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c200689a0 tx=0x7f5c100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e16dca1c584c879 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a61056700 1 --2- 172.21.15.57:0/786278907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 0x7f7a5c163d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009910 tx=0x7f7a4c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf20d2906b72983 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b1bfff700 1 -- 172.21.15.57:0/1787479489 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bbe0 con 0x7f6b2406c800 2023-10-07T23:21:30.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 0x7f8b240cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b24155040 0x7f8b240cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5b7fe700 1 -- 172.21.15.57:0/786278907 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540017f0 con 0x7f7a5c0af530 2023-10-07T23:21:30.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b240c6880 con 0x7f8b2406abd0 2023-10-07T23:21:30.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b240cae70 con 0x7f8b24155040 2023-10-07T23:21:30.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b28980700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2406b780 con 0x7f8b24154c70 2023-10-07T23:21:30.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1e7fc700 1 -- 172.21.15.57:0/1882689892 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011630 con 0x7f5c2015ceb0 2023-10-07T23:21:30.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b23fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8b2406abd0 0x7f8b2406b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60492/0 (socket says 172.21.15.57:60492) 2023-10-07T23:21:30.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b23fff700 1 -- 172.21.15.57:0/3740751842 learned_addr learned my addr 172.21.15.57:0/3740751842 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1e7fc700 1 -- 172.21.15.57:0/1882689892 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10011790 con 0x7f5c2015ceb0 2023-10-07T23:21:30.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c1e7fc700 1 -- 172.21.15.57:0/1882689892 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011b50 con 0x7f5c2015ceb0 2023-10-07T23:21:30.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5b7fe700 1 -- 172.21.15.57:0/786278907 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c011420 con 0x7f7a5c0af530 2023-10-07T23:21:30.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3740751842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 0x7f8b240cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a5b7fe700 1 -- 172.21.15.57:0/786278907 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0117a0 con 0x7f7a5c0af530 2023-10-07T23:21:30.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd6ffd700 1 -- 172.21.15.57:0/3676775939 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00be60 con 0x7fbdd806d440 2023-10-07T23:21:30.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/1787479489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406c800 msgr2=0x7f6b24154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/1787479489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406c800 0x7f6b24154830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b240c7b30 tx=0x7f6b20009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd6ffd700 1 -- 172.21.15.57:0/3676775939 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc00de00 con 0x7fbdd806d440 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 -- 172.21.15.57:0/3740751842 >> v2:172.21.15.154:3300/0 conn(0x7f8b24155040 msgr2=0x7f8b240cc320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3740751842 >> v2:172.21.15.154:3300/0 conn(0x7f8b24155040 0x7f8b240cc320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 -- 172.21.15.57:0/3740751842 >> v1:172.21.15.57:6789/0 conn(0x7f8b2406abd0 legacy=0x7f8b2406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 -- 172.21.15.57:0/3740751842 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2406bc80 con 0x7f8b24154c70 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/1787479489 shutdown_connections 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/1787479489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406c800 0x7f6b24154830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/1787479489 >> v2:172.21.15.57:3300/0 conn(0x7f6b240ad3e0 0x7f6b2406d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/1787479489 >> 172.21.15.57:0/1787479489 conn(0x7f6b24068520 msgr2=0x7f6b24068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- 172.21.15.57:0/1882689892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 msgr2=0x7f5c2015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- 172.21.15.57:0/1882689892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 0x7f5c2015d330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c200689a0 tx=0x7f5c100091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 -- 172.21.15.57:0/786278907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 msgr2=0x7f7a5c163d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/786278907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 0x7f7a5c163d80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009910 tx=0x7f7a4c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3740751842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 0x7f8b240cbde0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b180090e0 tx=0x7f8b18009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487024a3532b5104 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- 172.21.15.57:0/1882689892 shutdown_connections 2023-10-07T23:21:30.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 -- 172.21.15.57:0/786278907 shutdown_connections 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/786278907 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c0af530 0x7f7a5c163d80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 -- 172.21.15.57:0/786278907 >> 172.21.15.57:0/786278907 conn(0x7f7a5c068520 msgr2=0x7f7a5c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 -- 172.21.15.57:0/786278907 shutdown_connections 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/1787479489 shutdown_connections 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- 172.21.15.57:0/1882689892 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c2015ceb0 0x7f5c2015d330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- 172.21.15.57:0/1882689892 >> v2:172.21.15.57:3300/0 conn(0x7f5c200cae30 0x7f5c200c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 --2- 172.21.15.57:0/1882689892 >> v2:172.21.15.154:3300/0 conn(0x7f5c200c8500 0x7f5c200c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f5c2603a700 1 -- 172.21.15.57:0/1882689892 >> 172.21.15.57:0/1882689892 conn(0x7f5c200670e0 msgr2=0x7f5c200674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f7a62ab9700 1 -- 172.21.15.57:0/786278907 wait complete. 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7fbdd6ffd700 1 -- 172.21.15.57:0/3676775939 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0196b0 con 0x7fbdd806d440 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- 172.21.15.57:0/3676775939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd806d440 msgr2=0x7fbdd80cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd806d440 0x7fbdd80cbda0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00bc50 tx=0x7fbdcc00b4f0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- 172.21.15.57:0/3676775939 shutdown_connections 2023-10-07T23:21:30.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd806d440 0x7fbdd80cbda0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 --2- 172.21.15.57:0/3676775939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- 172.21.15.57:0/3676775939 >> 172.21.15.57:0/3676775939 conn(0x7fbdd8068520 msgr2=0x7fbdd8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- 172.21.15.57:0/3676775939 shutdown_connections 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f8b227fc700 1 -- 172.21.15.57:0/3740751842 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800f750 con 0x7f8b24154c70 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f8b227fc700 1 -- 172.21.15.57:0/3740751842 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800f8b0 con 0x7f8b24154c70 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f8b227fc700 1 -- 172.21.15.57:0/3740751842 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800fc70 con 0x7f8b24154c70 2023-10-07T23:21:30.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.746+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/1787479489 wait complete. 2023-10-07T23:21:30.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 Processor -- start 2023-10-07T23:21:30.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- 172.21.15.57:0/1882689892 shutdown_connections 2023-10-07T23:21:30.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- 172.21.15.57:0/1882689892 wait complete. 2023-10-07T23:21:30.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f7a62ab9700 1 Processor -- start 2023-10-07T23:21:30.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 -- start start 2023-10-07T23:21:30.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 Processor -- start 2023-10-07T23:21:30.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- start start 2023-10-07T23:21:30.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- 172.21.15.57:0/3676775939 wait complete. 2023-10-07T23:21:30.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 Processor -- start 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c200cae30 0x7f5c200c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 0x7f5c2016f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c5b60 con 0x7f5c2015ceb0 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c4fc0 con 0x7f5c200cae30 2023-10-07T23:21:30.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c2603a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c3990 con 0x7f5c200c8500 2023-10-07T23:21:30.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7fbddd196700 1 -- start start 2023-10-07T23:21:30.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f8b28980700 1 -- 172.21.15.57:0/3740751842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 msgr2=0x7f8b240cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f8b28980700 1 --2- 172.21.15.57:0/3740751842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 0x7f8b240cbde0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b180090e0 tx=0x7f8b18009a30 comp rx=0 tx=0).stop 2023-10-07T23:21:30.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56860/0 (socket says 172.21.15.57:56860) 2023-10-07T23:21:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 -- 172.21.15.57:0/3993284072 learned_addr learned my addr 172.21.15.57:0/3993284072 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1effd700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c200cae30 0x7f5c200c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f7a62ab9700 1 -- start start 2023-10-07T23:21:30.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 0x7f5c2016f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2406c800 0x7f6b2414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 0x7f6b240b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240c36e0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240afa80 con 0x7f6b2406ce80 2023-10-07T23:21:30.778 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2b8b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240af220 con 0x7f6b2406c800 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 -- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c200cae30 msgr2=0x7f5c200c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c200cae30 0x7f5c200c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 -- 172.21.15.57:0/3993284072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 msgr2=0x7f5c2016f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 0x7f5c2016f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 -- 172.21.15.57:0/3993284072 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18009ae0 con 0x7f5c200c8500 2023-10-07T23:21:30.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1ffff700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 0x7f5c2016f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b28e4e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 0x7f6b240b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b29e50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b28e4e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 0x7f6b240b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36982/0 (socket says 172.21.15.57:36982) 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b28e4e700 1 -- 172.21.15.57:0/4061320666 learned_addr learned my addr 172.21.15.57:0/4061320666 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b29e50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58462/0 (socket says 172.21.15.57:58462) 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f6b2964f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2406c800 0x7f6b2414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.750+0000 7f5c1f7fe700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400b840 tx=0x7f5c14009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 0x7fbdd814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 -- 172.21.15.57:0/3740751842 shutdown_connections 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 --2- 172.21.15.57:0/3740751842 >> v2:172.21.15.154:3300/0 conn(0x7f8b24155040 0x7f8b240cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 --2- 172.21.15.57:0/3740751842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b24154c70 0x7f8b240cbde0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 -- 172.21.15.57:0/3740751842 >> 172.21.15.57:0/3740751842 conn(0x7f8b24068520 msgr2=0x7f8b24068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 -- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2406c800 msgr2=0x7f6b2414d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2406c800 0x7f6b2414d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 -- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 msgr2=0x7f6b240b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 0x7f6b240b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 -- 172.21.15.57:0/4061320666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200094f0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a5c06cf10 0x7f7a5c0c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 0x7f7a5c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c158fb0 con 0x7f7a5c0af530 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c0c3120 con 0x7f7a5c06cf10 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c157670 con 0x7f7a5c158be0 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80af5f0 con 0x7fbdd8155920 2023-10-07T23:21:30.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd814a360 con 0x7fbdd81554a0 2023-10-07T23:21:30.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80aefc0 con 0x7fbdd81550d0 2023-10-07T23:21:30.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58468/0 (socket says 172.21.15.57:58468) 2023-10-07T23:21:30.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 -- 172.21.15.57:0/173660479 learned_addr learned my addr 172.21.15.57:0/173660479 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a60855700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a5c06cf10 0x7f7a5c0c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddc995700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddc995700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58472/0 (socket says 172.21.15.57:58472) 2023-10-07T23:21:30.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 0x7fbdd814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddc995700 1 -- 172.21.15.57:0/4256738347 learned_addr learned my addr 172.21.15.57:0/4256738347 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 -- 172.21.15.57:0/3740751842 shutdown_connections 2023-10-07T23:21:30.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b29e50700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b240cb910 tx=0x7f6b20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a61056700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 0x7f7a5c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 -- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 msgr2=0x7f7a5c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 0x7f7a5c0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 -- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a5c06cf10 msgr2=0x7f7a5c0c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a5c06cf10 0x7f7a5c0c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 -- 172.21.15.57:0/173660479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009420 con 0x7f7a5c0af530 2023-10-07T23:21:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400f510 con 0x7f5c200c8500 2023-10-07T23:21:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd7fff700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c2603a700 1 -- 172.21.15.57:0/3993284072 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c10009420 con 0x7f5c200c8500 2023-10-07T23:21:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c2603a700 1 -- 172.21.15.57:0/3993284072 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c200c5510 con 0x7f5c200c8500 2023-10-07T23:21:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a61056700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 0x7f7a5c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a5bfff700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a540019e0 tx=0x7f7a54007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 -- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 msgr2=0x7fbdd80c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 -- 172.21.15.57:0/4256738347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 msgr2=0x7fbdd814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 -- 172.21.15.57:0/4256738347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc00b790 con 0x7fbdd81554a0 2023-10-07T23:21:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddc995700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400ce80 con 0x7f7a5c0af530 2023-10-07T23:21:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54004dc0 con 0x7f7a5c0af530 2023-10-07T23:21:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400abb0 con 0x7f7a5c0af530 2023-10-07T23:21:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5c159200 con 0x7f7a5c0af530 2023-10-07T23:21:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c1597c0 con 0x7f7a5c0af530 2023-10-07T23:21:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdd77fe700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 0x7fbdd814baa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80c63f0 tx=0x7fbdcc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400f670 con 0x7f5c200c8500 2023-10-07T23:21:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400fa30 con 0x7f5c200c8500 2023-10-07T23:21:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 -- 172.21.15.57:0/3740751842 wait complete. 2023-10-07T23:21:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bbe0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20011420 con 0x7f6b2414d9e0 2023-10-07T23:21:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b240af4c0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b240c3370 con 0x7f6b2414d9e0 2023-10-07T23:21:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200117e0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 Processor -- start 2023-10-07T23:21:30.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00be60 con 0x7fbdd81554a0 2023-10-07T23:21:30.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc010680 con 0x7fbdd81554a0 2023-10-07T23:21:30.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc010a20 con 0x7fbdd81554a0 2023-10-07T23:21:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80c53c0 con 0x7fbdd81554a0 2023-10-07T23:21:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd80af280 con 0x7fbdd81554a0 2023-10-07T23:21:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c2603a700 1 -- 172.21.15.57:0/3993284072 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20068700 con 0x7f5c200c8500 2023-10-07T23:21:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7bf2501700 1 Processor -- start 2023-10-07T23:21:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f7bf2501700 1 -- start start 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f8b28980700 1 -- start start 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b2406abd0 0x7f8b24149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b24155040 0x7f8b240c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24157b30 con 0x7f8b24154c70 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24149fa0 con 0x7f8b24155040 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b28980700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b24156a90 con 0x7f8b2406abd0 2023-10-07T23:21:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7fc522be6700 1 Processor -- start 2023-10-07T23:21:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58476/0 (socket says 172.21.15.57:58476) 2023-10-07T23:21:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 -- 172.21.15.57:0/3204033150 learned_addr learned my addr 172.21.15.57:0/3204033150 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b23fff700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b24155040 0x7f8b240c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b2406abd0 0x7f8b24149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7a54007150 con 0x7f7a5c0af530 2023-10-07T23:21:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf2501700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7bec0c8ac0 0x7f7bec0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb80052b0 con 0x7fbdd81554a0 2023-10-07T23:21:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 -- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b2406abd0 msgr2=0x7f8b24149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b2406abd0 0x7f8b24149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 -- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b24155040 msgr2=0x7f8b240c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b24155040 0x7f8b240c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 -- 172.21.15.57:0/3204033150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18009cb0 con 0x7f8b24154c70 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f8b22ffd700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b600 tx=0x7f8b140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b040052b0 con 0x7f6b2414d9e0 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6b20011c10 con 0x7f6b2414d9e0 2023-10-07T23:21:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 -- start start 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.754+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c1401a400 con 0x7f5c200c8500 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f5c077fe700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 0x7f5c0c062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c1407c3a0 con 0x7f5c200c8500 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f5c1effd700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 0x7f5c0c062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f5c1effd700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 0x7f5c0c062b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c2006b210 tx=0x7f5c1800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5c1599b0 con 0x7f7a5c0af530 2023-10-07T23:21:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a58ff9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a5407c3c0 con 0x7f7a5c0af530 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a60855700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf2501700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec0cbf70 con 0x7f7bec0cc400 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf2501700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec0c72a0 con 0x7f7bec0c6990 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf2501700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec1619e0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7beb7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7bec0cc400 0x7f7bec0c83b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60502/0 (socket says 172.21.15.57:60502) 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7beb7fe700 1 -- 172.21.15.57:0/2197172907 learned_addr learned my addr 172.21.15.57:0/2197172907 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7beaffd700 1 -- 172.21.15.57:0/2197172907 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320910722 0 0) 0x7f7bec0cbf70 con 0x7f7bec0cc400 2023-10-07T23:21:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7beaffd700 1 -- 172.21.15.57:0/2197172907 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd4003570 con 0x7f7bec0cc400 2023-10-07T23:21:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 Processor -- start 2023-10-07T23:21:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf0a9e700 1 --2- 172.21.15.57:0/2197172907 >> v2:172.21.15.57:3300/0 conn(0x7f7bec0c8ac0 0x7f7bec0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f6b197fa700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 0x7f6b10062c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbdcc005a00 con 0x7fbdd81554a0 2023-10-07T23:21:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b2007e080 con 0x7f6b2414d9e0 2023-10-07T23:21:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc51c15ced0 0x7fc51c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c06bbf0 con 0x7fc51c0c8900 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c06b020 con 0x7fc51c15ced0 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc522be6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c06b2c0 con 0x7fc51c161a70 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7a60855700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 0x7f7a48062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ab0 tx=0x7f7a50009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf0a9e700 1 -- 172.21.15.57:0/2197172907 >> v1:172.21.15.154:6789/0 conn(0x7f7bec0c6990 legacy=0x7f7bec160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf0a9e700 1 -- 172.21.15.57:0/2197172907 >> v1:172.21.15.57:6789/0 conn(0x7f7bec0cc400 legacy=0x7f7bec0c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf0a9e700 1 -- 172.21.15.57:0/2197172907 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec161cc0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f7bf0a9e700 1 --2- 172.21.15.57:0/2197172907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0c6d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7be00098e0 tx=0x7f7be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aecd594f9eebd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc51c15ced0 0x7fc51c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc51c15ced0 0x7fc51c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58488/0 (socket says 172.21.15.57:58488) 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 -- 172.21.15.57:0/3655117869 learned_addr learned my addr 172.21.15.57:0/3655117869 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc5213e3700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fbdbffff700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 0x7fbdc4062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbdcc014070 con 0x7fbdd81554a0 2023-10-07T23:21:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 -- start start 2023-10-07T23:21:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7f6b2964f700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 0x7f6b10062c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fbdd7fff700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 0x7fbdc4062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 -- 172.21.15.57:0/3655117869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 msgr2=0x7fc51c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 -- 172.21.15.57:0/3655117869 >> v1:172.21.15.57:6789/0 conn(0x7fc51c0c8900 legacy=0x7fc51c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 -- 172.21.15.57:0/3655117869 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c06b5a0 con 0x7fc51c15ced0 2023-10-07T23:21:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7fc521be4700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c15ced0 0x7fc51c06c080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5180099b0 tx=0x7fc518009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7731f050826a6b47 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3b0155040 0x7ff3b00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b00c6880 con 0x7ff3b006abd0 2023-10-07T23:21:30.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00cae70 con 0x7ff3b0155040 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.758+0000 7ff3b52e5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b006b780 con 0x7ff3b0154c70 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3b0155040 0x7ff3b00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3b4ae4700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff3b006abd0 0x7ff3b006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:49846/0 (socket says 172.21.15.57:49846) 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3b4ae4700 1 -- 172.21.15.57:0/709586292 learned_addr learned my addr 172.21.15.57:0/709586292 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3affff700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f6b2964f700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 0x7f6b10062c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c0016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7fbdd7fff700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 0x7fbdc4062ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006fd0 tx=0x7fbdc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 -- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 msgr2=0x7ff3b00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 -- 172.21.15.57:0/709586292 >> v1:172.21.15.154:6789/0 conn(0x7ff3b006abd0 legacy=0x7ff3b006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 -- 172.21.15.57:0/709586292 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b006bc80 con 0x7ff3b0155040 2023-10-07T23:21:30.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7ff3af7fe700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155040 0x7ff3b00cc320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d100 tx=0x7ff3a800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d9ea96c20a1c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400f510 con 0x7f8b24154c70 2023-10-07T23:21:30.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b24156d70 con 0x7f8b24154c70 2023-10-07T23:21:30.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f7beaffd700 1 -- 172.21.15.57:0/2197172907 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000b640 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7fc520be2700 1 -- 172.21.15.57:0/3655117869 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518009ea0 con 0x7fc51c15ced0 2023-10-07T23:21:30.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b24157330 con 0x7f8b24154c70 2023-10-07T23:21:30.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.762+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7a5404ca30 con 0x7f7a5c0af530 2023-10-07T23:21:30.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.766+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b240711c0 con 0x7f8b24154c70 2023-10-07T23:21:30.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.766+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c20068700 con 0x7f5c200c8500 2023-10-07T23:21:30.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400f880 con 0x7f8b24154c70 2023-10-07T23:21:30.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400fa10 con 0x7f8b24154c70 2023-10-07T23:21:30.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc520be2700 1 -- 172.21.15.57:0/3655117869 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51800b860 con 0x7fc51c15ced0 2023-10-07T23:21:30.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc520be2700 1 -- 172.21.15.57:0/3655117869 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800bc00 con 0x7fc51c15ced0 2023-10-07T23:21:30.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6b2004e810 con 0x7f6b2414d9e0 2023-10-07T23:21:30.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbdcc04eaa0 con 0x7fbdd81554a0 2023-10-07T23:21:30.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3aeffd700 1 -- 172.21.15.57:0/709586292 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a8010070 con 0x7ff3b0155040 2023-10-07T23:21:30.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3aeffd700 1 -- 172.21.15.57:0/709586292 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800b2e0 con 0x7ff3b0155040 2023-10-07T23:21:30.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3aeffd700 1 -- 172.21.15.57:0/709586292 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a800dd70 con 0x7ff3b0155040 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 -- 172.21.15.57:0/3655117869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c15ced0 msgr2=0x7fc51c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c15ced0 0x7fc51c06c080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5180099b0 tx=0x7fc518009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 -- 172.21.15.57:0/3655117869 shutdown_connections 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 --2- 172.21.15.57:0/3655117869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c15ced0 0x7fc51c06c080 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5180099b0 tx=0x7fc518009250 comp rx=0 tx=0).stop 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7fc522be6700 1 -- 172.21.15.57:0/3655117869 >> 172.21.15.57:0/3655117869 conn(0x7fc51c0670e0 msgr2=0x7fc51c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 -- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155040 msgr2=0x7ff3b00cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155040 0x7ff3b00cc320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d100 tx=0x7ff3a800b990 comp rx=0 tx=0).stop 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 -- 172.21.15.57:0/709586292 shutdown_connections 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155040 0x7ff3b00cc320 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800d100 tx=0x7ff3a800b990 comp rx=0 tx=0).stop 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/709586292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 -- 172.21.15.57:0/709586292 >> 172.21.15.57:0/709586292 conn(0x7ff3b0068520 msgr2=0x7ff3b0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f7beaffd700 1 -- 172.21.15.57:0/2197172907 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be000b7a0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f7beaffd700 1 -- 172.21.15.57:0/2197172907 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000bb20 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f7bf2501700 1 -- 172.21.15.57:0/2197172907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0c8ac0 msgr2=0x7f7bec0c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f7bf2501700 1 --2- 172.21.15.57:0/2197172907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0c6d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7be00098e0 tx=0x7f7be00091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 -- 172.21.15.57:0/709586292 shutdown_connections 2023-10-07T23:21:30.815 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7ff3b52e5700 1 -- 172.21.15.57:0/709586292 wait complete. 2023-10-07T23:21:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8b1400fb70 con 0x7f8b24154c70 2023-10-07T23:21:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b137fe700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 0x7f8b080629d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b1407c910 con 0x7f8b24154c70 2023-10-07T23:21:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.770+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 0x7f8b080629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7f8b237fe700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 0x7f8b080629d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b240afa70 tx=0x7f8b18018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 Processor -- start 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 -- start start 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 0x7ff3b01426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 0x7ff3b0153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00aec20 con 0x7ff3b006abd0 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01568f0 con 0x7ff3b0155850 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b52e5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0155ce0 con 0x7ff3b0154c70 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 0x7ff3b01426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3affff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 0x7ff3b01426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58500/0 (socket says 172.21.15.57:58500) 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3affff700 1 -- 172.21.15.57:0/77811083 learned_addr learned my addr 172.21.15.57:0/77811083 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 0x7ff3b0153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3af7fe700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 -- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 msgr2=0x7ff3b00b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 -- 172.21.15.57:0/77811083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 msgr2=0x7ff3b01426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 0x7ff3b01426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 -- 172.21.15.57:0/77811083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a800bc30 con 0x7ff3b0155850 2023-10-07T23:21:30.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.774+0000 7ff3b4ae4700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 0x7ff3b0153cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c440 tx=0x7ff3a4007ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- 172.21.15.57:0/3655117869 shutdown_connections 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- 172.21.15.57:0/3655117869 wait complete. 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 Processor -- start 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7f7bf2501700 1 -- 172.21.15.57:0/2197172907 shutdown_connections 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7f7bf2501700 1 --2- 172.21.15.57:0/2197172907 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0c6d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7f7bf2501700 1 -- 172.21.15.57:0/2197172907 >> 172.21.15.57:0/2197172907 conn(0x7f7bec0670e0 msgr2=0x7f7bec0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:30.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7f7bf2501700 1 -- 172.21.15.57:0/2197172907 shutdown_connections 2023-10-07T23:21:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- start start 2023-10-07T23:21:30.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7f7bf2501700 1 -- 172.21.15.57:0/2197172907 wait complete. 2023-10-07T23:21:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4005900 con 0x7ff3b0155850 2023-10-07T23:21:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4005a60 con 0x7ff3b0155850 2023-10-07T23:21:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b0155fc0 con 0x7ff3b0155850 2023-10-07T23:21:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b0156580 con 0x7ff3b0155850 2023-10-07T23:21:30.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4005e00 con 0x7ff3b0155850 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 0x7fc51c145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc51c0c4f30 0x7fc51c0c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c0c5dc0 con 0x7fc51c0c8900 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c146800 con 0x7fc51c0c4f30 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.778+0000 7fc522be6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c159f70 con 0x7fc51c161a70 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc5213e3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc5213e3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c0c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56950/0 (socket says 172.21.15.57:56950) 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc5213e3700 1 -- 172.21.15.57:0/2097840486 learned_addr learned my addr 172.21.15.57:0/2097840486 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 0x7fc51c145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 Processor -- start 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 -- start start 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 -- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 msgr2=0x7fc51c0c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c0c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 -- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc51c0c4f30 msgr2=0x7fc51c0c19a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T23:21:30.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc51c0c4f30 0x7fc51c0c19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 -- 172.21.15.57:0/2097840486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180094f0 con 0x7fc51c0c8900 2023-10-07T23:21:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc521be4700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 0x7fc51c145ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc51c06a240 tx=0x7fc518017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b1404d020 con 0x7f8b24154c70 2023-10-07T23:21:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3900052b0 con 0x7ff3b0155850 2023-10-07T23:21:30.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800bbe0 con 0x7fc51c0c8900 2023-10-07T23:21:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff3a40072e0 con 0x7ff3b0155850 2023-10-07T23:21:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c0c5520 con 0x7fc51c0c8900 2023-10-07T23:21:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c0c5ae0 con 0x7fc51c0c8900 2023-10-07T23:21:30.826 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518011420 con 0x7fc51c0c8900 2023-10-07T23:21:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 0x7f7bec0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180117c0 con 0x7fc51c0c8900 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0cc400 0x7f7bec0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 0x7f7bec0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bebfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 0x7f7bec0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56962/0 (socket says 172.21.15.57:56962) 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7beb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7beb7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:37042/0 (socket says 172.21.15.57:37042) 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bebfff700 1 -- 172.21.15.57:0/4247406861 learned_addr learned my addr 172.21.15.57:0/4247406861 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:30.828 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec15b580 con 0x7f7bec0cc400 2023-10-07T23:21:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec15c0d0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec15c560 con 0x7f7bec0c6990 2023-10-07T23:21:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7ff397fff700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 0x7ff398062ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff3a407ba30 con 0x7ff3b0155850 2023-10-07T23:21:30.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7ff3affff700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 0x7ff398062ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.782+0000 7f7bf0a9e700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0cc400 0x7f7bec0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 -- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 msgr2=0x7f7bec0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 0x7f7bec0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 -- 172.21.15.57:0/4247406861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0cc400 msgr2=0x7f7bec0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0cc400 0x7f7bec0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009420 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7f7beb7fe700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c590 tx=0x7f7bdc007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc518011bf0 con 0x7fc51c0c8900 2023-10-07T23:21:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7fc511ffb700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 0x7fc508062b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc51807e0b0 con 0x7fc51c0c8900 2023-10-07T23:21:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7fc5213e3700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 0x7fc508062b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.786+0000 7fc5213e3700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 0x7fc508062b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0c2780 tx=0x7fc51400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.790+0000 7ff3affff700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 0x7ff398062ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009de0 tx=0x7ff3a00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc0052b0 con 0x7fc51c0c8900 2023-10-07T23:21:30.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff3a404c000 con 0x7ff3b0155850 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc005c00 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc005d80 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec0af020 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc007320 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.794+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec0aedf0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.798+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7bdc01b3b0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.798+0000 7f7bd3fff700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 0x7f7bd8062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.802+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bdc07bb70 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.798+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc51804e6a0 con 0x7fc51c0c8900 2023-10-07T23:21:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.802+0000 7f7bebfff700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 0x7f7bd8062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.802+0000 7f7bebfff700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 0x7f7bd8062ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bec15c880 tx=0x7f7be4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.810+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc0052b0 con 0x7f7bec0c8ac0 2023-10-07T23:21:30.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:30.814+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7bdc04c1e0 con 0x7f7bec0c8ac0 2023-10-07T23:21:31.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.082+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbdb8005830 con 0x7fbdd81554a0 2023-10-07T23:21:31.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.082+0000 7f5c2603a700 1 -- 172.21.15.57:0/3993284072 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5c20068700 con 0x7f5c200c8500 2023-10-07T23:21:31.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.082+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7a5c0711c0 con 0x7f7a5c0af530 2023-10-07T23:21:31.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.082+0000 7f7a58ff9700 1 -- 172.21.15.57:0/173660479 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a54050050 con 0x7f7a5c0af530 2023-10-07T23:21:31.090 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.082+0000 7f5c077fe700 1 -- 172.21.15.57:0/3993284072 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c20068700 con 0x7f5c200c8500 2023-10-07T23:21:31.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.086+0000 7fbdbffff700 1 -- 172.21.15.57:0/4256738347 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbdcc0520c0 con 0x7fbdd81554a0 2023-10-07T23:21:31.091 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.092 INFO:teuthology.orchestra.run.smithi057.stdout:60129542150 2023-10-07T23:21:31.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.086+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6b04005830 con 0x7f6b2414d9e0 2023-10-07T23:21:31.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.086+0000 7f6b197fa700 1 -- 172.21.15.57:0/4061320666 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b20015020 con 0x7f6b2414d9e0 2023-10-07T23:21:31.093 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 msgr2=0x7f7a48062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 0x7f7a48062a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ab0 tx=0x7f7a50009060 comp rx=0 tx=0).stop 2023-10-07T23:21:31.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 msgr2=0x7f7a5c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a540019e0 tx=0x7f7a54007e50 comp rx=0 tx=0).stop 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 shutdown_connections 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a5c158be0 0x7f7a5c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a5c0af530 0x7f7a5c0b3c40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a48060550 0x7f7a48062a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 --2- 172.21.15.57:0/173660479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a5c06cf10 0x7f7a5c0c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 >> 172.21.15.57:0/173660479 conn(0x7f7a5c068520 msgr2=0x7f7a5c0ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 shutdown_connections 2023-10-07T23:21:31.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f7a62ab9700 1 -- 172.21.15.57:0/173660479 wait complete. 2023-10-07T23:21:31.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 msgr2=0x7f6b10062c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 0x7f6b10062c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c0016f0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 msgr2=0x7f6b240bf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b240cb910 tx=0x7f6b20017000 comp rx=0 tx=0).stop 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 shutdown_connections 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6b2414d9e0 0x7f6b240bf070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6b10060780 0x7f6b10062c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b2406ce80 0x7f6b240b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 --2- 172.21.15.57:0/4061320666 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6b2406c800 0x7f6b2414d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 >> 172.21.15.57:0/4061320666 conn(0x7f6b24068520 msgr2=0x7f6b240c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 shutdown_connections 2023-10-07T23:21:31.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.090+0000 7f6b2b8b3700 1 -- 172.21.15.57:0/4061320666 wait complete. 2023-10-07T23:21:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 msgr2=0x7f5c0c062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 0x7f5c0c062b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c2006b210 tx=0x7f5c1800b040 comp rx=0 tx=0).stop 2023-10-07T23:21:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 msgr2=0x7f5c20176280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400b840 tx=0x7f5c14009250 comp rx=0 tx=0).stop 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 shutdown_connections 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c2015ceb0 0x7f5c2016f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c0c0606a0 0x7f5c0c062b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c200cae30 0x7f5c200c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 --2- 172.21.15.57:0/3993284072 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c200c8500 0x7f5c20176280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 >> 172.21.15.57:0/3993284072 conn(0x7f5c200670e0 msgr2=0x7f5c200ac730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 msgr2=0x7fbdc4062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 shutdown_connections 2023-10-07T23:21:31.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7f5c057fa700 1 -- 172.21.15.57:0/3993284072 wait complete. 2023-10-07T23:21:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 0x7fbdc4062ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8006fd0 tx=0x7fbdc8008040 comp rx=0 tx=0).stop 2023-10-07T23:21:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 msgr2=0x7fbdd814baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 0x7fbdd814baa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80c63f0 tx=0x7fbdcc01b000 comp rx=0 tx=0).stop 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 shutdown_connections 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdd8155920 0x7fbdd814a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbdc4060620 0x7fbdc4062ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdd81554a0 0x7fbdd814baa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 --2- 172.21.15.57:0/4256738347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdd81550d0 0x7fbdd80c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 >> 172.21.15.57:0/4256738347 conn(0x7fbdd8068520 msgr2=0x7fbdd80cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 shutdown_connections 2023-10-07T23:21:31.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.098+0000 7fbddd196700 1 -- 172.21.15.57:0/4256738347 wait complete. 2023-10-07T23:21:31.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-10-07T23:21:31.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.102+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b240cb580 con 0x7f8b24154c70 2023-10-07T23:21:31.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.102+0000 7f8b137fe700 1 -- 172.21.15.57:0/3204033150 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b14050640 con 0x7f8b24154c70 2023-10-07T23:21:31.110 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2023-10-07T23:21:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 msgr2=0x7f8b080629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 0x7f8b080629d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b240afa70 tx=0x7f8b18018000 comp rx=0 tx=0).stop 2023-10-07T23:21:31.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 msgr2=0x7f8b240c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b600 tx=0x7f8b140091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 shutdown_connections 2023-10-07T23:21:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8b08060520 0x7f8b080629d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b24155040 0x7f8b240c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b24154c70 0x7f8b240c4d30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 --2- 172.21.15.57:0/3204033150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8b2406abd0 0x7f8b24149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 >> 172.21.15.57:0/3204033150 conn(0x7f8b24068520 msgr2=0x7f8b24154310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 shutdown_connections 2023-10-07T23:21:31.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.106+0000 7f8b28980700 1 -- 172.21.15.57:0/3204033150 wait complete. 2023-10-07T23:21:31.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2023-10-07T23:21:31.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2023-10-07T23:21:31.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.114+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc4fc005830 con 0x7fc51c0c8900 2023-10-07T23:21:31.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.114+0000 7fc511ffb700 1 -- 172.21.15.57:0/2097840486 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc518015030 con 0x7fc51c0c8900 2023-10-07T23:21:31.120 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 msgr2=0x7fc508062b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 0x7fc508062b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc51c0c2780 tx=0x7fc51400b040 comp rx=0 tx=0).stop 2023-10-07T23:21:31.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 msgr2=0x7fc51c145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 0x7fc51c145ce0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc51c06a240 tx=0x7fc518017000 comp rx=0 tx=0).stop 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 shutdown_connections 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc508060670 0x7fc508062b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc51c0c4f30 0x7fc51c0c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc51c161a70 0x7fc51c0c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 --2- 172.21.15.57:0/2097840486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc51c0c8900 0x7fc51c145ce0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 >> 172.21.15.57:0/2097840486 conn(0x7fc51c0670e0 msgr2=0x7fc51c15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 shutdown_connections 2023-10-07T23:21:31.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7fc522be6700 1 -- 172.21.15.57:0/2097840486 wait complete. 2023-10-07T23:21:31.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2023-10-07T23:21:31.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.118+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff390005560 con 0x7ff3b0155850 2023-10-07T23:21:31.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff397fff700 1 -- 172.21.15.57:0/77811083 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3a404f620 con 0x7ff3b0155850 2023-10-07T23:21:31.126 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 msgr2=0x7ff398062ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 0x7ff398062ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009de0 tx=0x7ff3a00091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 msgr2=0x7ff3b0153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 0x7ff3b0153cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c440 tx=0x7ff3a4007ea0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 shutdown_connections 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff3980607f0 0x7ff398062ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3b0155850 0x7ff3b0153cd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3b0154c70 0x7ff3b00b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 --2- 172.21.15.57:0/77811083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3b006abd0 0x7ff3b01426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 >> 172.21.15.57:0/77811083 conn(0x7ff3b0068520 msgr2=0x7ff3b00670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 shutdown_connections 2023-10-07T23:21:31.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.122+0000 7ff3b52e5700 1 -- 172.21.15.57:0/77811083 wait complete. 2023-10-07T23:21:31.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2023-10-07T23:21:31.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.130+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7bcc005830 con 0x7f7bec0c8ac0 2023-10-07T23:21:31.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bd3fff700 1 -- 172.21.15.57:0/4247406861 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7bdc017020 con 0x7f7bec0c8ac0 2023-10-07T23:21:31.138 INFO:teuthology.orchestra.run.smithi057.stdout:64424509445 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 msgr2=0x7f7bd8062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 0x7f7bd8062ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bec15c880 tx=0x7f7be4009040 comp rx=0 tx=0).stop 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 msgr2=0x7f7bec0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c590 tx=0x7f7bdc007d50 comp rx=0 tx=0).stop 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 shutdown_connections 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7bec0cc400 0x7f7bec0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7bd8060620 0x7f7bd8062ad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7bec0c8ac0 0x7f7bec0b0460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 --2- 172.21.15.57:0/4247406861 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7bec0c6990 0x7f7bec0ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 >> 172.21.15.57:0/4247406861 conn(0x7f7bec0670e0 msgr2=0x7f7bec160de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 shutdown_connections 2023-10-07T23:21:31.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:31.134+0000 7f7bf2501700 1 -- 172.21.15.57:0/4247406861 wait complete. 2023-10-07T23:21:31.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2023-10-07T23:21:31.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2023-10-07T23:21:32.109 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-07T23:21:32.111 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-07T23:21:32.118 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-07T23:21:32.118 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-07T23:21:32.126 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-07T23:21:32.138 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-07T23:21:32.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-07T23:21:32.150 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-07T23:21:32.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 Processor -- start 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 -- start start 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa0a006ce80 0x7fa0a006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 0x7fa0a0154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a006b320 con 0x7fa0a006ade0 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a006b600 con 0x7fa0a006ce80 2023-10-07T23:21:32.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa0a4cd8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a006b8e0 con 0x7fa0a006a960 2023-10-07T23:21:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.234+0000 7fa09dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 0x7fa0a0154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 0x7fa0a0154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56968/0 (socket says 172.21.15.57:56968) 2023-10-07T23:21:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09dd9b700 1 -- 172.21.15.57:0/2911013272 learned_addr learned my addr 172.21.15.57:0/2911013272 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 --2- 172.21.15.57:0/2911013272 >> v2:172.21.15.57:3300/0 conn(0x7fa0a006ce80 0x7fa0a006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 -- 172.21.15.57:0/2911013272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 msgr2=0x7fa0a0154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 --2- 172.21.15.57:0/2911013272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 0x7fa0a0154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 -- 172.21.15.57:0/2911013272 >> v1:172.21.15.57:6789/0 conn(0x7fa0a006ade0 legacy=0x7fa0a0154d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 -- 172.21.15.57:0/2911013272 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a006bbc0 con 0x7fa0a006ce80 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09e59c700 1 --2- 172.21.15.57:0/2911013272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006ce80 0x7fa0a006d250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0900088f0 tx=0x7fa090008210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e1d3d676b16d6f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09d59a700 1 -- 172.21.15.57:0/2911013272 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090010070 con 0x7fa0a006ce80 2023-10-07T23:21:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa09d59a700 1 -- 172.21.15.57:0/2911013272 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000d9e0 con 0x7fa0a006ce80 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/2911013272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006ce80 msgr2=0x7fa0a006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 --2- 172.21.15.57:0/2911013272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006ce80 0x7fa0a006d250 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0900088f0 tx=0x7fa090008210 comp rx=0 tx=0).stop 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/2911013272 shutdown_connections 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 --2- 172.21.15.57:0/2911013272 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006a960 0x7fa0a0154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 --2- 172.21.15.57:0/2911013272 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006ce80 0x7fa0a006d250 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/2911013272 >> 172.21.15.57:0/2911013272 conn(0x7fa0a0068520 msgr2=0x7fa0a0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/2911013272 shutdown_connections 2023-10-07T23:21:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.238+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/2911013272 wait complete. 2023-10-07T23:21:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 Processor -- start 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 -- start start 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0a006ade0 0x7fa0a0169d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006ce80 0x7fa0a016bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c1720 con 0x7fa0a006a960 2023-10-07T23:21:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c0b10 con 0x7fa0a006ade0 2023-10-07T23:21:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa0a4cd8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00c0df0 con 0x7fa0a006ce80 2023-10-07T23:21:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa09e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa09e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58534/0 (socket says 172.21.15.57:58534) 2023-10-07T23:21:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa09e59c700 1 -- 172.21.15.57:0/4277614243 learned_addr learned my addr 172.21.15.57:0/4277614243 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.242+0000 7fa09dd9b700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0a006ade0 0x7fa0a0169d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 -- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006ce80 msgr2=0x7fa0a016bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006ce80 0x7fa0a016bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 -- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0a006ade0 msgr2=0x7fa0a0169d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0a006ade0 0x7fa0a0169d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 -- 172.21.15.57:0/4277614243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900084b0 con 0x7fa0a006a960 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa09e59c700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0900030c0 tx=0x7fa09001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090017ae0 con 0x7fa0a006a960 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090017c60 con 0x7fa0a006a960 2023-10-07T23:21:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000b0c0 con 0x7fa0a006a960 2023-10-07T23:21:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/4277614243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a00c10d0 con 0x7fa0a006a960 2023-10-07T23:21:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.246+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/4277614243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a00c20f0 con 0x7fa0a006a960 2023-10-07T23:21:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa09001d070 con 0x7fa0a006a960 2023-10-07T23:21:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/4277614243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0840052b0 con 0x7fa0a006a960 2023-10-07T23:21:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa08e7fc700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 0x7fa088062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa09007e250 con 0x7fa0a006a960 2023-10-07T23:21:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa09dd9b700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 0x7fa088062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.250+0000 7fa09dd9b700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 0x7fa088062b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01793c0 tx=0x7fa09800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.258+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa0900522d0 con 0x7fa0a006a960 2023-10-07T23:21:32.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 Processor -- start 2023-10-07T23:21:32.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 Processor -- start 2023-10-07T23:21:32.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 -- start start 2023-10-07T23:21:32.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 Processor -- start 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1f0815a180 0x7f1f080c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f08152f10 0x7f1f08153370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0815d600 con 0x7f1f0815a180 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f081538b0 con 0x7f1f08152f10 2023-10-07T23:21:32.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0cdcd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0815c9f0 con 0x7f1f0806ca30 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 -- start start 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc54806ce80 0x7fc54806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548154830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc54806ade0 0x7fc548154d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54806b320 con 0x7fc54806ade0 2023-10-07T23:21:32.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54806b600 con 0x7fc54806ce80 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54f980700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54806b8e0 con 0x7fc54806a960 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f05d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1f0815a180 0x7f1f080c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56982/0 (socket says 172.21.15.57:56982) 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 -- 172.21.15.57:0/4053354127 learned_addr learned my addr 172.21.15.57:0/4053354127 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f06d9d700 1 --2- 172.21.15.57:0/4053354127 >> v2:172.21.15.154:3300/0 conn(0x7f1f08152f10 0x7f1f08153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc54806ade0 0x7fc548154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc54806ade0 0x7fc548154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58558/0 (socket says 172.21.15.57:58558) 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 -- 172.21.15.57:0/3654448392 learned_addr learned my addr 172.21.15.57:0/3654448392 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54cf1b700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 Processor -- start 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 -- start start 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ca80cbd40 0x7f8ca8154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80ca200 con 0x7f8ca80cb970 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca80ca7c0 con 0x7f8ca8155010 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8cad5d9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80c69a0 con 0x7f8ca80cbd40 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ca80cbd40 0x7f8ca8154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8ca80cbd40 0x7f8ca8154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:37066/0 (socket says 172.21.15.57:37066) 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 -- 172.21.15.57:0/1881264648 learned_addr learned my addr 172.21.15.57:0/1881264648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca6ffd700 1 --1- 172.21.15.57:0/1881264648 >> v1:172.21.15.57:6789/0 conn(0x7f8ca80cb970 0x7f8ca80cac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60524/0 (socket says 172.21.15.57:60524) 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 -- 172.21.15.57:0/3654448392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 msgr2=0x7fc548154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 -- 172.21.15.57:0/3654448392 >> v2:172.21.15.154:3300/0 conn(0x7fc54806ce80 msgr2=0x7fc54806d250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-07T23:21:32.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 --2- 172.21.15.57:0/3654448392 >> v2:172.21.15.154:3300/0 conn(0x7fc54806ce80 0x7fc54806d250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc54df1d700 1 -- 172.21.15.57:0/3654448392 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54806bbc0 con 0x7fc54806ade0 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 -- 172.21.15.57:0/4053354127 >> v2:172.21.15.154:3300/0 conn(0x7f1f08152f10 msgr2=0x7f1f08153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 --2- 172.21.15.57:0/4053354127 >> v2:172.21.15.154:3300/0 conn(0x7f1f08152f10 0x7f1f08153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 -- 172.21.15.57:0/4053354127 >> v2:172.21.15.57:3300/0 conn(0x7f1f0815a180 msgr2=0x7f1f080c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 --2- 172.21.15.57:0/4053354127 >> v2:172.21.15.57:3300/0 conn(0x7f1f0815a180 0x7f1f080c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f1f0659c700 1 -- 172.21.15.57:0/4053354127 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0815ccd0 con 0x7f1f0806ca30 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 -- start start 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0340c8500 0x7fc0340c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0340cae30 0x7fc0340c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0340ca9a0 con 0x7fc03415ceb0 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340c9ab0 con 0x7fc0340cae30 2023-10-07T23:21:32.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7fc038f1f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340c9d90 con 0x7fc0340c8500 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 -- 172.21.15.57:0/1881264648 >> v1:172.21.15.154:6789/0 conn(0x7f8ca8155010 legacy=0x7f8ca80c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 -- 172.21.15.57:0/1881264648 >> v1:172.21.15.57:6789/0 conn(0x7f8ca80cb970 legacy=0x7f8ca80cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.262+0000 7f8ca67fc700 1 -- 172.21.15.57:0/1881264648 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80c8db0 con 0x7f8ca80cbd40 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc032d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fc03415ceb0 0x7fc03415d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60532/0 (socket says 172.21.15.57:60532) 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc032d9d700 1 -- 172.21.15.57:0/3796460033 learned_addr learned my addr 172.21.15.57:0/3796460033 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 --2- 172.21.15.57:0/3796460033 >> v2:172.21.15.154:3300/0 conn(0x7fc0340cae30 0x7fc0340c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc03259c700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0340c8500 0x7fc0340c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54df1d700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548154d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5480cb910 tx=0x7fc5440082d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a69514ea738f34 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8ca67fc700 1 --2- 172.21.15.57:0/1881264648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca80cbd40 0x7f8ca8154ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099b0 tx=0x7f8c9c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca05e464e99713e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0659c700 1 --2- 172.21.15.57:0/4053354127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f080c8b80 tx=0x7f1ef8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bfcf6c66994175 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc53ffff700 1 -- 172.21.15.57:0/3654448392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5440126e0 con 0x7fc54806ade0 2023-10-07T23:21:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0559a700 1 -- 172.21.15.57:0/4053354127 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8009e70 con 0x7f1f0806ca30 2023-10-07T23:21:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0559a700 1 -- 172.21.15.57:0/4053354127 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800b860 con 0x7f1f0806ca30 2023-10-07T23:21:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0559a700 1 -- 172.21.15.57:0/4053354127 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800bc00 con 0x7f1f0806ca30 2023-10-07T23:21:32.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 -- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0340c8500 msgr2=0x7fc0340c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0340c8500 0x7fc0340c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 -- 172.21.15.57:0/3796460033 >> v1:172.21.15.57:6789/0 conn(0x7fc03415ceb0 legacy=0x7fc03415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 -- 172.21.15.57:0/3796460033 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0340ca070 con 0x7fc0340cae30 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8ca5ffb700 1 -- 172.21.15.57:0/1881264648 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00b860 con 0x7f8ca80cbd40 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc53ffff700 1 -- 172.21.15.57:0/3654448392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544012840 con 0x7fc54806ade0 2023-10-07T23:21:32.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc031d9b700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc0340c6d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098e0 tx=0x7fc0280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950a4ff1dbe65660 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8ca5ffb700 1 -- 172.21.15.57:0/1881264648 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00b9c0 con 0x7f8ca80cbd40 2023-10-07T23:21:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8ca5ffb700 1 -- 172.21.15.57:0/1881264648 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00bd40 con 0x7f8ca80cbd40 2023-10-07T23:21:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/4053354127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 msgr2=0x7f1f0806ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/4053354127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f080c8b80 tx=0x7f1ef8009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc53ffff700 1 -- 172.21.15.57:0/3654448392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544012be0 con 0x7fc54806ade0 2023-10-07T23:21:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc03159a700 1 -- 172.21.15.57:0/3796460033 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800b640 con 0x7fc0340cae30 2023-10-07T23:21:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 -- 172.21.15.57:0/1881264648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca80cbd40 msgr2=0x7f8ca8154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/1881264648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca80cbd40 0x7f8ca8154ad0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099b0 tx=0x7f8c9c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/4053354127 shutdown_connections 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/4053354127 >> v2:172.21.15.154:3300/0 conn(0x7f1f08152f10 0x7f1f08153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/4053354127 >> v2:172.21.15.57:3300/0 conn(0x7f1f0815a180 0x7f1f080c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/4053354127 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f0806ca30 0x7f1f0806ce00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/4053354127 >> 172.21.15.57:0/4053354127 conn(0x7f1f08068520 msgr2=0x7f1f08068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 -- 172.21.15.57:0/3654448392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 msgr2=0x7fc548154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548154d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc5480cb910 tx=0x7fc5440082d0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/4053354127 shutdown_connections 2023-10-07T23:21:32.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 -- 172.21.15.57:0/1881264648 shutdown_connections 2023-10-07T23:21:32.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/1881264648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca80cbd40 0x7f8ca8154ad0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 -- 172.21.15.57:0/1881264648 >> 172.21.15.57:0/1881264648 conn(0x7f8ca80670e0 msgr2=0x7f8ca80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc03159a700 1 -- 172.21.15.57:0/3796460033 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800b7a0 con 0x7fc0340cae30 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 -- 172.21.15.57:0/3654448392 shutdown_connections 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548154d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 --2- 172.21.15.57:0/3654448392 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 --2- 172.21.15.57:0/3654448392 >> v2:172.21.15.154:3300/0 conn(0x7fc54806ce80 0x7fc54806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 -- 172.21.15.57:0/3654448392 >> 172.21.15.57:0/3654448392 conn(0x7fc548068520 msgr2=0x7fc548068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/4053354127 wait complete. 2023-10-07T23:21:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc03159a700 1 -- 172.21.15.57:0/3796460033 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800bb20 con 0x7fc0340cae30 2023-10-07T23:21:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 -- 172.21.15.57:0/3654448392 shutdown_connections 2023-10-07T23:21:32.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 -- 172.21.15.57:0/1881264648 shutdown_connections 2023-10-07T23:21:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc54f980700 1 -- 172.21.15.57:0/3654448392 wait complete. 2023-10-07T23:21:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 Processor -- start 2023-10-07T23:21:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 Processor -- start 2023-10-07T23:21:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f1f0cdcd700 1 -- start start 2023-10-07T23:21:32.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 0x7f1f0816c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f08152f10 0x7f1f08149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 0x7f1f080a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c4840 con 0x7f1f0815a180 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080a8430 con 0x7f1f0806ca30 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0816cae0 con 0x7f1f08152f10 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 -- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 msgr2=0x7fc0340c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc0340c6d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc0280098e0 tx=0x7fc0280091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 -- 172.21.15.57:0/3796460033 shutdown_connections 2023-10-07T23:21:32.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc0340c6d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 --2- 172.21.15.57:0/3796460033 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc0340c8500 0x7fc0340c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7fc038f1f700 1 -- 172.21.15.57:0/3796460033 >> 172.21.15.57:0/3796460033 conn(0x7fc0340670e0 msgr2=0x7fc0340674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 -- 172.21.15.57:0/3796460033 shutdown_connections 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 -- 172.21.15.57:0/3796460033 wait complete. 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.266+0000 7f8cad5d9700 1 -- 172.21.15.57:0/1881264648 wait complete. 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f06d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 0x7f1f080a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f06d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 0x7f1f080a7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58572/0 (socket says 172.21.15.57:58572) 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f06d9d700 1 -- 172.21.15.57:0/1362718604 learned_addr learned my addr 172.21.15.57:0/1362718604 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 0x7f1f0816c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f05d9b700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f08152f10 0x7f1f08149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 -- start start 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 Processor -- start 2023-10-07T23:21:32.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 -- start start 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 -- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f08152f10 msgr2=0x7f1f08149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f08152f10 0x7f1f08149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 -- 172.21.15.57:0/1362718604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 msgr2=0x7f1f080a7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 0x7f1f080a7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 -- 172.21.15.57:0/1362718604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80094f0 con 0x7f1f0806ca30 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548169050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ca80cbd40 0x7f8ca80b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 0x7f8ca80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80bf0e0 con 0x7f8ca80cb970 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80bff40 con 0x7f8ca8155010 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8cad5d9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80bf330 con 0x7f8ca80cbd40 2023-10-07T23:21:32.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc54806ce80 0x7fc5480c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54816a5b0 con 0x7fc54806ade0 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5481699a0 con 0x7fc54806ce80 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54f980700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc548169c80 con 0x7fc54806a960 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0659c700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 0x7f1f0816c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806ea90 tx=0x7f1ef8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 Processor -- start 2023-10-07T23:21:32.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58582/0 (socket says 172.21.15.57:58582) 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 -- 172.21.15.57:0/2991538860 learned_addr learned my addr 172.21.15.57:0/2991538860 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58576/0 (socket says 172.21.15.57:58576) 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 -- 172.21.15.57:0/422900779 learned_addr learned my addr 172.21.15.57:0/422900779 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca77fe700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 0x7f8ca80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54d71c700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548169050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca67fc700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ca80cbd40 0x7f8ca80b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800bbe0 con 0x7f1f0806ca30 2023-10-07T23:21:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8011420 con 0x7f1f0806ca30 2023-10-07T23:21:32.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080a7e70 con 0x7f1f0806ca30 2023-10-07T23:21:32.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54df1d700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc54806ce80 0x7fc5480c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080a8200 con 0x7f1f0806ca30 2023-10-07T23:21:32.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80117a0 con 0x7f1f0806ca30 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 -- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ca80cbd40 msgr2=0x7f8ca80b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ca80cbd40 0x7f8ca80b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 -- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 msgr2=0x7f8ca80b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 0x7f8ca80b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 -- 172.21.15.57:0/422900779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0094f0 con 0x7f8ca80cb970 2023-10-07T23:21:32.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca77fe700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 0x7f8ca80b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 -- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 msgr2=0x7fc548169050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548169050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 -- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc54806ce80 msgr2=0x7fc5480c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc54806ce80 0x7fc5480c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc54cf1b700 1 -- 172.21.15.57:0/2991538860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544008570 con 0x7fc54806ade0 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 -- start start 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 0x7fc0341406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7fc038f1f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc03414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc038f1f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03415ceb0 0x7fc03414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc038f1f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034146f90 con 0x7fc0340c8500 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc038f1f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034146680 con 0x7fc0340cae30 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc038f1f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0340c5330 con 0x7fc03415ceb0 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.270+0000 7f8ca6ffd700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000cd60 tx=0x7f8ca0001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc54cf1b700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc540009ef0 tx=0x7fc54000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc03259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 0x7fc0341406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc03259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 0x7fc0341406f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58598/0 (socket says 172.21.15.57:58598) 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc03259c700 1 -- 172.21.15.57:0/2413498749 learned_addr learned my addr 172.21.15.57:0/2413498749 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc03414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc032d9d700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03415ceb0 0x7fc03414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0005df0 con 0x7f8ca80cb970 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000a920 con 0x7f8ca80cb970 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000ace0 con 0x7f8ca80cb970 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54000cb70 con 0x7fc54806ade0 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54000f040 con 0x7fc54806ade0 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540014480 con 0x7fc54806ade0 2023-10-07T23:21:32.299 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec0052b0 con 0x7f1f0806ca30 2023-10-07T23:21:32.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca80bf610 con 0x7f8ca80cb970 2023-10-07T23:21:32.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca80bfbd0 con 0x7f8ca80cb970 2023-10-07T23:21:32.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc548169f60 con 0x7fc54806ade0 2023-10-07T23:21:32.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc548169590 con 0x7fc54806ade0 2023-10-07T23:21:32.300 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 -- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03415ceb0 msgr2=0x7fc03414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03415ceb0 0x7fc03414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 -- 172.21.15.57:0/2413498749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 msgr2=0x7fc0341406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 0x7fc0341406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 -- 172.21.15.57:0/2413498749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009420 con 0x7fc0340cae30 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc5400147f0 con 0x7fc54806ade0 2023-10-07T23:21:32.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc031d9b700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc03414cb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc028016000 tx=0x7fc028017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca80711c0 con 0x7f8ca80cb970 2023-10-07T23:21:32.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 0x7fc52c062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5280052b0 con 0x7fc54806ade0 2023-10-07T23:21:32.302 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8ca0007300 con 0x7f8ca80cb970 2023-10-07T23:21:32.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc54007d040 con 0x7fc54806ade0 2023-10-07T23:21:32.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.274+0000 7fc54d71c700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 0x7fc52c062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1ef8011bd0 con 0x7f1f0806ca30 2023-10-07T23:21:32.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f8c8effd700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 0x7f8c94062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ca007c810 con 0x7f8ca80cb970 2023-10-07T23:21:32.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f1ef67fc700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 0x7f1ee8062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.304 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f8ca67fc700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 0x7f8c94062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ef807db50 con 0x7f1f0806ca30 2023-10-07T23:21:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f1f05d9b700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 0x7f1ee8062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f8ca67fc700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 0x7f8c94062ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099b0 tx=0x7f8c9c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7fc54d71c700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 0x7fc52c062ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc538009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.278+0000 7f1f05d9b700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 0x7f1ee8062ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f080a47f0 tx=0x7f1f0000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02801f070 con 0x7fc0340cae30 2023-10-07T23:21:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028002f30 con 0x7fc0340cae30 2023-10-07T23:21:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034146940 con 0x7fc0340cae30 2023-10-07T23:21:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800b910 con 0x7fc0340cae30 2023-10-07T23:21:32.306 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03406bfa0 con 0x7fc0340cae30 2023-10-07T23:21:32.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ef804e150 con 0x7f1f0806ca30 2023-10-07T23:21:32.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.282+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc54004d6b0 con 0x7fc54806ade0 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc028023070 con 0x7fc0340cae30 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0340711c0 con 0x7fc0340cae30 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc0227fc700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 0x7fc014062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc02807de60 con 0x7fc0340cae30 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc03259c700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 0x7fc014062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.286+0000 7fc03259c700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 0x7fc014062a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0099f0 tx=0x7fc02c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.290+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ca004cf60 con 0x7f8ca80cb970 2023-10-07T23:21:32.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.294+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc02804e3b0 con 0x7fc0340cae30 2023-10-07T23:21:32.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.306+0000 7f817b4ae700 1 Processor -- start 2023-10-07T23:21:32.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.306+0000 7f817b4ae700 1 -- start start 2023-10-07T23:21:32.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f81740cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8174155040 0x7f81740cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740c6880 con 0x7f817406abd0 2023-10-07T23:21:32.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81740cae70 con 0x7f8174155040 2023-10-07T23:21:32.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817406b780 con 0x7f8174154c70 2023-10-07T23:21:32.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8174155040 0x7f81740cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8174155040 0x7f81740cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:37118/0 (socket says 172.21.15.57:37118) 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 -- 172.21.15.57:0/2951156104 learned_addr learned my addr 172.21.15.57:0/2951156104 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817acad700 1 --1- 172.21.15.57:0/2951156104 >> v1:172.21.15.154:6789/0 conn(0x7f817406abd0 0x7f817406b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:49860/0 (socket says 172.21.15.57:49860) 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817a4ac700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f81740cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 -- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 msgr2=0x7f81740cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f81740cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 -- 172.21.15.57:0/2951156104 >> v1:172.21.15.154:6789/0 conn(0x7f817406abd0 legacy=0x7f817406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 Processor -- start 2023-10-07T23:21:32.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 -- 172.21.15.57:0/2951156104 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817406bc80 con 0x7f8174155040 2023-10-07T23:21:32.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 -- start start 2023-10-07T23:21:32.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f8179cab700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8174155040 0x7f81740cc320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81700099e0 tx=0x7f8170009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8bd78199b53822 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c0c7320 con 0x7fe98c06bc20 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98c06acf0 con 0x7fe98c06b7d0 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe994662700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c06afd0 con 0x7fe98c06c9a0 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7ff64359e700 1 Processor -- start 2023-10-07T23:21:32.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7ff64359e700 1 -- start start 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f81794aa700 1 -- 172.21.15.57:0/2951156104 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170009ed0 con 0x7f8174155040 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f81794aa700 1 -- 172.21.15.57:0/2951156104 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817000b760 con 0x7f8174155040 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f81794aa700 1 -- 172.21.15.57:0/2951156104 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817000bb00 con 0x7f8174155040 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8174155040 msgr2=0x7f81740cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8174155040 0x7f81740cc320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81700099e0 tx=0x7f8170009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- 172.21.15.57:0/2951156104 shutdown_connections 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8174155040 0x7f81740cc320 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f81700099e0 tx=0x7f8170009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 --2- 172.21.15.57:0/2951156104 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f81740cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- 172.21.15.57:0/2951156104 >> 172.21.15.57:0/2951156104 conn(0x7f8174068520 msgr2=0x7f8174068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe9923fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe9923fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57048/0 (socket says 172.21.15.57:57048) 2023-10-07T23:21:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7fe9923fe700 1 -- 172.21.15.57:0/4239702801 learned_addr learned my addr 172.21.15.57:0/4239702801 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- 172.21.15.57:0/2951156104 shutdown_connections 2023-10-07T23:21:32.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.310+0000 7f817b4ae700 1 -- 172.21.15.57:0/2951156104 wait complete. 2023-10-07T23:21:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9923fe700 1 -- 172.21.15.57:0/4239702801 >> v1:172.21.15.154:6789/0 conn(0x7fe98c06b7d0 legacy=0x7fe98c0ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9923fe700 1 -- 172.21.15.57:0/4239702801 >> v1:172.21.15.57:6789/0 conn(0x7fe98c06bc20 legacy=0x7fe98c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9923fe700 1 -- 172.21.15.57:0/4239702801 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c06b2b0 con 0x7fe98c06c9a0 2023-10-07T23:21:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9923fe700 1 --2- 172.21.15.57:0/4239702801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099e0 tx=0x7fe988009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90acd146410df1d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7ff64359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9913fc700 1 -- 172.21.15.57:0/4239702801 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800b820 con 0x7fe98c06c9a0 2023-10-07T23:21:32.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe9913fc700 1 -- 172.21.15.57:0/4239702801 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800b980 con 0x7fe98c06c9a0 2023-10-07T23:21:32.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 Processor -- start 2023-10-07T23:21:32.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 -- start start 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 -- 172.21.15.57:0/4239702801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 msgr2=0x7fe98c06cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 --2- 172.21.15.57:0/4239702801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099e0 tx=0x7fe988009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 -- 172.21.15.57:0/4239702801 shutdown_connections 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 --2- 172.21.15.57:0/4239702801 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06c9a0 0x7fe98c06cd70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 -- 172.21.15.57:0/4239702801 >> 172.21.15.57:0/4239702801 conn(0x7fe98c068740 msgr2=0x7fe98c068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7ff64259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7ff64259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57060/0 (socket says 172.21.15.57:57060) 2023-10-07T23:21:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7ff64259c700 1 -- 172.21.15.57:0/2694940939 learned_addr learned my addr 172.21.15.57:0/2694940939 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7ff641d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff644157a30 0x7ff6440c82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60550/0 (socket says 172.21.15.57:60550) 2023-10-07T23:21:32.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 -- 172.21.15.57:0/4239702801 shutdown_connections 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f817406abd0 0x7f81741484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f817416a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817416b550 con 0x7f8174169b60 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174148a10 con 0x7f817406abd0 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817b4ae700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817416a940 con 0x7f8174154c70 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58614/0 (socket says 172.21.15.57:58614) 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 -- 172.21.15.57:0/3840425974 learned_addr learned my addr 172.21.15.57:0/3840425974 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817a4ac700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f817406abd0 0x7f81741484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f8179cab700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f817416a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 -- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 msgr2=0x7f817416a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f817416a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 -- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f817406abd0 msgr2=0x7f81741484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f817406abd0 0x7f81741484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7f817acad700 1 -- 172.21.15.57:0/3840425974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700094f0 con 0x7f8174169b60 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f8179cab700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f817416a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.314+0000 7fe994662700 1 -- 172.21.15.57:0/4239702801 wait complete. 2023-10-07T23:21:32.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 Processor -- start 2023-10-07T23:21:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f817acad700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f816c00ce00 tx=0x7f816c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 -- start start 2023-10-07T23:21:32.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff644161b70 con 0x7ff644157a30 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff644160f60 con 0x7ff6440c89d0 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff644160d50 con 0x7ff644157660 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64259c700 1 -- 172.21.15.57:0/2694940939 >> v1:172.21.15.154:6789/0 conn(0x7ff6440c89d0 legacy=0x7ff64406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64259c700 1 -- 172.21.15.57:0/2694940939 >> v1:172.21.15.57:6789/0 conn(0x7ff644157a30 legacy=0x7ff6440c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64259c700 1 -- 172.21.15.57:0/2694940939 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644161240 con 0x7ff644157660 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c005df0 con 0x7f8174169b60 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c00a920 con 0x7f8174169b60 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c00acc0 con 0x7f8174169b60 2023-10-07T23:21:32.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f817b4ae700 1 -- 172.21.15.57:0/3840425974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817416ac20 con 0x7f8174169b60 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64259c700 1 --2- 172.21.15.57:0/2694940939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6340099e0 tx=0x7ff634009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d49a16b251d832 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06b7d0 0x7fe98c0a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe98c06c9a0 0x7fe98c175920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0a6160 con 0x7fe98c06c9a0 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0a5550 con 0x7fe98c06bc20 2023-10-07T23:21:32.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe994662700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0a5830 con 0x7fe98c06b7d0 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:37136/0 (socket says 172.21.15.57:37136) 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 -- 172.21.15.57:0/1966981782 learned_addr learned my addr 172.21.15.57:0/1966981782 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 -- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06b7d0 msgr2=0x7fe98c0a65f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06b7d0 0x7fe98c0a65f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 -- 172.21.15.57:0/1966981782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe98c06c9a0 msgr2=0x7fe98c175920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64159a700 1 -- 172.21.15.57:0/2694940939 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400b860 con 0x7ff644157660 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64159a700 1 -- 172.21.15.57:0/2694940939 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63400b9c0 con 0x7ff644157660 2023-10-07T23:21:32.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64159a700 1 -- 172.21.15.57:0/2694940939 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63400bd60 con 0x7ff644157660 2023-10-07T23:21:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 msgr2=0x7ff6440ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 --2- 172.21.15.57:0/2694940939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6340099e0 tx=0x7ff634009250 comp rx=0 tx=0).stop 2023-10-07T23:21:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7f817b4ae700 1 -- 172.21.15.57:0/3840425974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817416b1e0 con 0x7f8174169b60 2023-10-07T23:21:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe98c06c9a0 0x7fe98c175920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 -- 172.21.15.57:0/1966981782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880094f0 con 0x7fe98c06bc20 2023-10-07T23:21:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 shutdown_connections 2023-10-07T23:21:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 --2- 172.21.15.57:0/2694940939 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff644157660 0x7ff6440ca300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 >> 172.21.15.57:0/2694940939 conn(0x7ff6440670e0 msgr2=0x7ff6440674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 shutdown_connections 2023-10-07T23:21:32.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7ff64359e700 1 -- 172.21.15.57:0/2694940939 wait complete. 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe991bfd700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe98400cb10 tx=0x7fe9840017b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400a760 con 0x7fe98c06bc20 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98400a8e0 con 0x7fe98c06bc20 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.318+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400aca0 con 0x7fe98c06bc20 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe994662700 1 -- 172.21.15.57:0/1966981782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c0a5b10 con 0x7fe98c06bc20 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe994662700 1 -- 172.21.15.57:0/1966981782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c0c4860 con 0x7fe98c06bc20 2023-10-07T23:21:32.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 Processor -- start 2023-10-07T23:21:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f816c007300 con 0x7f8174169b60 2023-10-07T23:21:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7f816a7fc700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 0x7f8158062a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 -- start start 2023-10-07T23:21:32.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 0x7ff6440c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 0x7ff6440c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f816c07ca50 con 0x7f8174169b60 2023-10-07T23:21:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7f817a4ac700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 0x7f8158062a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7f817a4ac700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 0x7f8158062a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8164009eb0 tx=0x7f816400d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 0x7ff6440c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 0x7ff6440c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57102/0 (socket says 172.21.15.57:57102) 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58634/0 (socket says 172.21.15.57:58634) 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64259c700 1 -- 172.21.15.57:0/4243531750 learned_addr learned my addr 172.21.15.57:0/4243531750 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64415d0f0 con 0x7ff644157a30 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64414f190 con 0x7ff644157660 2023-10-07T23:21:32.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440c5360 con 0x7ff6440c89d0 2023-10-07T23:21:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe984003030 con 0x7fe98c06bc20 2023-10-07T23:21:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe9827fc700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 0x7fe970062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe98407c8e0 con 0x7fe98c06bc20 2023-10-07T23:21:32.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7fe9923fe700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 0x7fe970062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff641d9b700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 0x7ff6440c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 -- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 msgr2=0x7ff6440c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 0x7ff6440c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 -- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 msgr2=0x7ff6440c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 0x7ff6440c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340094f0 con 0x7ff644157a30 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff641d9b700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 0x7ff6440c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:32.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.322+0000 7ff642d9d700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c830 tx=0x7ff63c001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7fe9923fe700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 0x7fe970062a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099e0 tx=0x7fe988016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c012630 con 0x7ff644157a30 2023-10-07T23:21:32.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c012790 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c012b50 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6440c5640 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6440c5c00 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff63c01a3a0 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 0x7ff624062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.326+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff63c07c8f0 con 0x7ff644157a30 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.330+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817416b6c0 con 0x7f8174169b60 2023-10-07T23:21:32.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.330+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0647b0 con 0x7fe98c06bc20 2023-10-07T23:21:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.330+0000 7ff64259c700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 0x7ff624062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.334+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64415d2a0 con 0x7ff644157a30 2023-10-07T23:21:32.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.334+0000 7ff64259c700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 0x7ff624062a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff634009930 tx=0x7ff634017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.338+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f816c04d020 con 0x7f8174169b60 2023-10-07T23:21:32.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.338+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe98404ced0 con 0x7fe98c06bc20 2023-10-07T23:21:32.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.350+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff64415d2a0 con 0x7ff644157a30 2023-10-07T23:21:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.558+0000 7fa0a4cd8700 1 -- 172.21.15.57:0/4277614243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa084005830 con 0x7fa0a006a960 2023-10-07T23:21:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa08e7fc700 1 -- 172.21.15.57:0/4277614243 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa090051e60 con 0x7fa0a006a960 2023-10-07T23:21:32.568 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 msgr2=0x7fa088062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 0x7fa088062b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0a01793c0 tx=0x7fa09800b040 comp rx=0 tx=0).stop 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 msgr2=0x7fa0a00c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0900030c0 tx=0x7fa09001b000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 shutdown_connections 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0a006ce80 0x7fa0a016bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa0880606d0 0x7fa088062b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.562+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0a006ade0 0x7fa0a0169d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.566+0000 7fa07ffff700 1 --2- 172.21.15.57:0/4277614243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0a006a960 0x7fa0a00c1bb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.566+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 >> 172.21.15.57:0/4277614243 conn(0x7fa0a0068520 msgr2=0x7fa0a00caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.566+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 shutdown_connections 2023-10-07T23:21:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.566+0000 7fa07ffff700 1 -- 172.21.15.57:0/4277614243 wait complete. 2023-10-07T23:21:32.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2023-10-07T23:21:32.584 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.582+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8ca80c9d00 con 0x7f8ca80cb970 2023-10-07T23:21:32.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.582+0000 7f8c8effd700 1 -- 172.21.15.57:0/422900779 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ca0050580 con 0x7f8ca80cb970 2023-10-07T23:21:32.590 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 msgr2=0x7f8c94062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 0x7f8c94062ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099b0 tx=0x7f8c9c019000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 msgr2=0x7f8ca8153d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000cd60 tx=0x7f8ca0001b40 comp rx=0 tx=0).stop 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 shutdown_connections 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8c94060620 0x7f8c94062ad0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca8155010 0x7f8ca80b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ca80cbd40 0x7f8ca80b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 --2- 172.21.15.57:0/422900779 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ca80cb970 0x7f8ca8153d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 >> 172.21.15.57:0/422900779 conn(0x7f8ca80670e0 msgr2=0x7f8ca80abd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 shutdown_connections 2023-10-07T23:21:32.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.586+0000 7f8cad5d9700 1 -- 172.21.15.57:0/422900779 wait complete. 2023-10-07T23:21:32.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.590+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc0340ca350 con 0x7fc0340cae30 2023-10-07T23:21:32.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.590+0000 7fc0227fc700 1 -- 172.21.15.57:0/2413498749 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc0280519d0 con 0x7fc0340cae30 2023-10-07T23:21:32.597 INFO:teuthology.orchestra.run.smithi057.stdout:60129542151 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 msgr2=0x7fc014062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 0x7fc014062a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0099f0 tx=0x7fc02c00b040 comp rx=0 tx=0).stop 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 msgr2=0x7fc03414cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc03414cb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc028016000 tx=0x7fc028017000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 shutdown_connections 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc03415ceb0 0x7fc03414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc0140605d0 0x7fc014062a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0340cae30 0x7fc03414cb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 --2- 172.21.15.57:0/2413498749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc0340c8500 0x7fc0341406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 >> 172.21.15.57:0/2413498749 conn(0x7fc0340670e0 msgr2=0x7fc03406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 shutdown_connections 2023-10-07T23:21:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.594+0000 7fc038f1f700 1 -- 172.21.15.57:0/2413498749 wait complete. 2023-10-07T23:21:32.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1eec005560 con 0x7f1f0806ca30 2023-10-07T23:21:32.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1ef67fc700 1 -- 172.21.15.57:0/1362718604 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ef8028020 con 0x7f1f0806ca30 2023-10-07T23:21:32.602 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2023-10-07T23:21:32.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc528005830 con 0x7fc54806ade0 2023-10-07T23:21:32.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7fc53d7fa700 1 -- 172.21.15.57:0/2991538860 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc540018020 con 0x7fc54806ade0 2023-10-07T23:21:32.606 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.606 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 msgr2=0x7f1ee8062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 0x7f1ee8062ab0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f080a47f0 tx=0x7f1f0000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 msgr2=0x7f1f0816c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 0x7f1f0816c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806ea90 tx=0x7f1ef8017000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 shutdown_connections 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1f0815a180 0x7f1f080a7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1f08152f10 0x7f1f08149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee8060600 0x7f1ee8062ab0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 --2- 172.21.15.57:0/1362718604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f0806ca30 0x7f1f0816c5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.598+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 >> 172.21.15.57:0/1362718604 conn(0x7f1f08068520 msgr2=0x7f1f080cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 shutdown_connections 2023-10-07T23:21:32.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7f1f0cdcd700 1 -- 172.21.15.57:0/1362718604 wait complete. 2023-10-07T23:21:32.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 msgr2=0x7fc52c062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.608 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 0x7fc52c062ad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc538009060 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 msgr2=0x7fc548153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc540009ef0 tx=0x7fc54000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 shutdown_connections 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc52c060620 0x7fc52c062ad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc54806ce80 0x7fc5480c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc54806ade0 0x7fc548153c90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 --2- 172.21.15.57:0/2991538860 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc54806a960 0x7fc548169050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 >> 172.21.15.57:0/2991538860 conn(0x7fc548068520 msgr2=0x7fc5480acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 shutdown_connections 2023-10-07T23:21:32.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.602+0000 7fc54f980700 1 -- 172.21.15.57:0/2991538860 wait complete. 2023-10-07T23:21:32.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2023-10-07T23:21:32.616 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2023-10-07T23:21:32.621 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2023-10-07T23:21:32.622 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.618+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81740711c0 con 0x7f8174169b60 2023-10-07T23:21:32.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.618+0000 7f816a7fc700 1 -- 172.21.15.57:0/3840425974 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f816c050640 con 0x7f8174169b60 2023-10-07T23:21:32.625 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 msgr2=0x7f8158062a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 0x7f8158062a80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8164009eb0 tx=0x7f816400d040 comp rx=0 tx=0).stop 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 msgr2=0x7f817416b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f816c00ce00 tx=0x7f816c001b80 comp rx=0 tx=0).stop 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 shutdown_connections 2023-10-07T23:21:32.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8174169b60 0x7f817416b9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8174154c70 0x7f817416a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f81580605d0 0x7f8158062a80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 --2- 172.21.15.57:0/3840425974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f817406abd0 0x7f81741484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 >> 172.21.15.57:0/3840425974 conn(0x7f8174068520 msgr2=0x7f8174152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 shutdown_connections 2023-10-07T23:21:32.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.622+0000 7f8157fff700 1 -- 172.21.15.57:0/3840425974 wait complete. 2023-10-07T23:21:32.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.630+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe98c0c7600 con 0x7fe98c06bc20 2023-10-07T23:21:32.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.630+0000 7fe9827fc700 1 -- 172.21.15.57:0/1966981782 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe9840504f0 con 0x7fe98c06bc20 2023-10-07T23:21:32.636 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2023-10-07T23:21:32.637 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 msgr2=0x7fe970062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 0x7fe970062a00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9880099e0 tx=0x7fe988016000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 msgr2=0x7fe98c0c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe98400cb10 tx=0x7fe9840017b0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 shutdown_connections 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe98c06c9a0 0x7fe98c175920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe970060550 0x7fe970062a00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe98c06bc20 0x7fe98c0c4320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 --2- 172.21.15.57:0/1966981782 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe98c06b7d0 0x7fe98c0a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 >> 172.21.15.57:0/1966981782 conn(0x7fe98c068740 msgr2=0x7fe98c0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 shutdown_connections 2023-10-07T23:21:32.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7fe96ffff700 1 -- 172.21.15.57:0/1966981782 wait complete. 2023-10-07T23:21:32.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff64415d2a0 con 0x7ff644157a30 2023-10-07T23:21:32.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.634+0000 7ff6327fc700 1 -- 172.21.15.57:0/4243531750 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff64415d2a0 con 0x7ff644157a30 2023-10-07T23:21:32.640 INFO:teuthology.orchestra.run.smithi057.stdout:64424509446 2023-10-07T23:21:32.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 msgr2=0x7ff624062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 0x7ff624062a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff634009930 tx=0x7ff634017000 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 msgr2=0x7ff64416d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00c830 tx=0x7ff63c001690 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 shutdown_connections 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff644157a30 0x7ff64416d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff624060550 0x7ff624062a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644157660 0x7ff6440c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 --2- 172.21.15.57:0/4243531750 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff6440c89d0 0x7ff6440c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 >> 172.21.15.57:0/4243531750 conn(0x7ff6440670e0 msgr2=0x7ff6440c1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 shutdown_connections 2023-10-07T23:21:32.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.638+0000 7ff64359e700 1 -- 172.21.15.57:0/4243531750 wait complete. 2023-10-07T23:21:32.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2023-10-07T23:21:32.648 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2023-10-07T23:21:32.652 DEBUG:teuthology.parallel:result is None 2023-10-07T23:21:32.652 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-07T23:21:32.652 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:21:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 Processor -- start 2023-10-07T23:21:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 -- start start 2023-10-07T23:21:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa070157d90 0x7fa0700c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07006ae80 con 0x7fa0700c8750 2023-10-07T23:21:32.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07015b050 con 0x7fa070151090 2023-10-07T23:21:32.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa07731d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07015ae60 con 0x7fa070157d90 2023-10-07T23:21:32.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa076b1c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa0700c8750 0x7fa07006b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60564/0 (socket says 172.21.15.57:60564) 2023-10-07T23:21:32.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa076b1c700 1 -- 172.21.15.57:0/3655306704 learned_addr learned my addr 172.21.15.57:0/3655306704 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:32.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075b1a700 1 --2- 172.21.15.57:0/3655306704 >> v2:172.21.15.154:3300/0 conn(0x7fa070157d90 0x7fa0700c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199918507 0 0) 0x7fa07006ae80 con 0x7fa0700c8750 2023-10-07T23:21:32.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003570 con 0x7fa0700c8750 2023-10-07T23:21:32.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075b1a700 1 -- 172.21.15.57:0/3655306704 >> v1:172.21.15.154:6789/0 conn(0x7fa070151090 legacy=0x7fa07015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075b1a700 1 -- 172.21.15.57:0/3655306704 >> v1:172.21.15.57:6789/0 conn(0x7fa0700c8750 legacy=0x7fa07006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945545630 0 0) 0x7fa05c003570 con 0x7fa0700c8750 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075b1a700 1 -- 172.21.15.57:0/3655306704 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07015b330 con 0x7fa070157d90 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075b1a700 1 --2- 172.21.15.57:0/3655306704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070157d90 0x7fa0700c8210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098e0 tx=0x7fa0640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2848292091eb870 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680017f0 con 0x7fa070157d90 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa064011420 con 0x7fa070157d90 2023-10-07T23:21:32.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.742+0000 7fa075319700 1 -- 172.21.15.57:0/3655306704 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640117a0 con 0x7fa070157d90 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- 172.21.15.57:0/3655306704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070157d90 msgr2=0x7fa0700c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 --2- 172.21.15.57:0/3655306704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070157d90 0x7fa0700c8210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0640098e0 tx=0x7fa0640091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- 172.21.15.57:0/3655306704 shutdown_connections 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 --2- 172.21.15.57:0/3655306704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070157d90 0x7fa0700c8210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- 172.21.15.57:0/3655306704 >> 172.21.15.57:0/3655306704 conn(0x7fa0700670e0 msgr2=0x7fa0700674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- 172.21.15.57:0/3655306704 shutdown_connections 2023-10-07T23:21:32.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- 172.21.15.57:0/3655306704 wait complete. 2023-10-07T23:21:32.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 Processor -- start 2023-10-07T23:21:32.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- start start 2023-10-07T23:21:32.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0700c8750 0x7fa07014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070151090 0x7fa0700af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700cae30 con 0x7fa070157d90 2023-10-07T23:21:32.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700ca220 con 0x7fa070151090 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.746+0000 7fa07731d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0700ca500 con 0x7fa0700c8750 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58638/0 (socket says 172.21.15.57:58638) 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 -- 172.21.15.57:0/1568352596 learned_addr learned my addr 172.21.15.57:0/1568352596 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa075b1a700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070151090 0x7fa0700af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa07631b700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0700c8750 0x7fa07014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 -- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0700c8750 msgr2=0x7fa07014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0700c8750 0x7fa07014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 -- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070151090 msgr2=0x7fa0700af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070151090 0x7fa0700af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 -- 172.21.15.57:0/1568352596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064009420 con 0x7fa070157d90 2023-10-07T23:21:32.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa076b1c700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa06800cb90 tx=0x7fa068007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068004dc0 con 0x7fa070157d90 2023-10-07T23:21:32.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800a5d0 con 0x7fa070157d90 2023-10-07T23:21:32.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0700ca7e0 con 0x7fa070157d90 2023-10-07T23:21:32.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800a990 con 0x7fa070157d90 2023-10-07T23:21:32.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0700cb2c0 con 0x7fa070157d90 2023-10-07T23:21:32.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa068007150 con 0x7fa070157d90 2023-10-07T23:21:32.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.750+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0700cafa0 con 0x7fa070157d90 2023-10-07T23:21:32.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.754+0000 7fa0627fc700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 0x7fa054062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:32.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.754+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa06807c2d0 con 0x7fa070157d90 2023-10-07T23:21:32.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.754+0000 7fa07631b700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 0x7fa054062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:32.759 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.754+0000 7fa07631b700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 0x7fa054062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa06c003aa0 tx=0x7fa06c000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:32.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.758+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa06804c8a0 con 0x7fa070157d90 2023-10-07T23:21:32.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.914+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0700711c0 con 0x7fa054060550 2023-10-07T23:21:32.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.930+0000 7fa0627fc700 1 -- 172.21.15.57:0/1568352596 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264419 (secure 0 0 0) 0x7fa0700711c0 con 0x7fa054060550 2023-10-07T23:21:32.938 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:32.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 msgr2=0x7fa054062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 0x7fa054062a00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa06c003aa0 tx=0x7fa06c000bc0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 msgr2=0x7fa0700af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa06800cb90 tx=0x7fa068007ca0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 shutdown_connections 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa070157d90 0x7fa0700af790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa054060550 0x7fa054062a00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa070151090 0x7fa0700af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 --2- 172.21.15.57:0/1568352596 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0700c8750 0x7fa07014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 >> 172.21.15.57:0/1568352596 conn(0x7fa0700670e0 msgr2=0x7fa0700c07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 shutdown_connections 2023-10-07T23:21:32.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:32.934+0000 7fa07731d700 1 -- 172.21.15.57:0/1568352596 wait complete. 2023-10-07T23:21:32.942 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:21:32.953 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-07T23:21:31.071729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397736,"kb_used_data":3048,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576984,"statfs":{"total":773094113280,"available":770638831616,"internally_reserved":0,"allocated":3121152,"data_stored":1830396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.139358"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824034+0000","last_change":"2023-10-07T23:21:25.817169+0000","last_active":"2023-10-07T23:21:25.824034+0000","last_peered":"2023-10-07T23:21:25.824034+0000","last_clean":"2023-10-07T23:21:25.824034+0000","last_became_active":"2023-10-07T23:21:25.816152+0000","last_became_peered":"2023-10-07T23:21:25.816152+0000","last_unstale":"2023-10-07T23:21:25.824034+0000","last_undegraded":"2023-10-07T23:21:25.824034+0000","last_fullsized":"2023-10-07T23:21:25.824034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820528+0000","last_change":"2023-10-07T23:21:25.817564+0000","last_active":"2023-10-07T23:21:25.820528+0000","last_peered":"2023-10-07T23:21:25.820528+0000","last_clean":"2023-10-07T23:21:25.820528+0000","last_became_active":"2023-10-07T23:21:25.817149+0000","last_became_peered":"2023-10-07T23:21:25.817149+0000","last_unstale":"2023-10-07T23:21:25.820528+0000","last_undegraded":"2023-10-07T23:21:25.820528+0000","last_fullsized":"2023-10-07T23:21:25.820528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:32.392634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823467+0000","last_change":"2023-10-07T23:21:25.813864+0000","last_active":"2023-10-07T23:21:25.823467+0000","last_peered":"2023-10-07T23:21:25.823467+0000","last_clean":"2023-10-07T23:21:25.823467+0000","last_became_active":"2023-10-07T23:21:25.813627+0000","last_became_peered":"2023-10-07T23:21:25.813627+0000","last_unstale":"2023-10-07T23:21:25.823467+0000","last_undegraded":"2023-10-07T23:21:25.823467+0000","last_fullsized":"2023-10-07T23:21:25.823467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822927+0000","last_change":"2023-10-07T23:21:25.817746+0000","last_active":"2023-10-07T23:21:25.822927+0000","last_peered":"2023-10-07T23:21:25.822927+0000","last_clean":"2023-10-07T23:21:25.822927+0000","last_became_active":"2023-10-07T23:21:25.817419+0000","last_became_peered":"2023-10-07T23:21:25.817419+0000","last_unstale":"2023-10-07T23:21:25.822927+0000","last_undegraded":"2023-10-07T23:21:25.822927+0000","last_fullsized":"2023-10-07T23:21:25.822927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824136+0000","last_change":"2023-10-07T23:21:25.817774+0000","last_active":"2023-10-07T23:21:25.824136+0000","last_peered":"2023-10-07T23:21:25.824136+0000","last_clean":"2023-10-07T23:21:25.824136+0000","last_became_active":"2023-10-07T23:21:25.817279+0000","last_became_peered":"2023-10-07T23:21:25.817279+0000","last_unstale":"2023-10-07T23:21:25.824136+0000","last_undegraded":"2023-10-07T23:21:25.824136+0000","last_fullsized":"2023-10-07T23:21:25.824136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824103+0000","last_change":"2023-10-07T23:21:25.821044+0000","last_active":"2023-10-07T23:21:25.824103+0000","last_peered":"2023-10-07T23:21:25.824103+0000","last_clean":"2023-10-07T23:21:25.824103+0000","last_became_active":"2023-10-07T23:21:25.820744+0000","last_became_peered":"2023-10-07T23:21:25.820744+0000","last_unstale":"2023-10-07T23:21:25.824103+0000","last_undegraded":"2023-10-07T23:21:25.824103+0000","last_fullsized":"2023-10-07T23:21:25.824103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.815381+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.815050+0000","last_became_peered":"2023-10-07T23:21:25.815050+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824873+0000","last_change":"2023-10-07T23:21:25.817306+0000","last_active":"2023-10-07T23:21:25.824873+0000","last_peered":"2023-10-07T23:21:25.824873+0000","last_clean":"2023-10-07T23:21:25.824873+0000","last_became_active":"2023-10-07T23:21:25.816989+0000","last_became_peered":"2023-10-07T23:21:25.816989+0000","last_unstale":"2023-10-07T23:21:25.824873+0000","last_undegraded":"2023-10-07T23:21:25.824873+0000","last_fullsized":"2023-10-07T23:21:25.824873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823714+0000","last_change":"2023-10-07T23:21:25.815302+0000","last_active":"2023-10-07T23:21:25.823714+0000","last_peered":"2023-10-07T23:21:25.823714+0000","last_clean":"2023-10-07T23:21:25.823714+0000","last_became_active":"2023-10-07T23:21:25.814105+0000","last_became_peered":"2023-10-07T23:21:25.814105+0000","last_unstale":"2023-10-07T23:21:25.823714+0000","last_undegraded":"2023-10-07T23:21:25.823714+0000","last_fullsized":"2023-10-07T23:21:25.823714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:37.211418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.805674+0000","last_change":"2023-10-07T23:21:25.800296+0000","last_active":"2023-10-07T23:21:25.805674+0000","last_peered":"2023-10-07T23:21:25.805674+0000","last_clean":"2023-10-07T23:21:25.805674+0000","last_became_active":"2023-10-07T23:21:25.799882+0000","last_became_peered":"2023-10-07T23:21:25.799882+0000","last_unstale":"2023-10-07T23:21:25.805674+0000","last_undegraded":"2023-10-07T23:21:25.805674+0000","last_fullsized":"2023-10-07T23:21:25.805674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806603+0000","last_change":"2023-10-07T23:21:25.803198+0000","last_active":"2023-10-07T23:21:25.806603+0000","last_peered":"2023-10-07T23:21:25.806603+0000","last_clean":"2023-10-07T23:21:25.806603+0000","last_became_active":"2023-10-07T23:21:25.802597+0000","last_became_peered":"2023-10-07T23:21:25.802597+0000","last_unstale":"2023-10-07T23:21:25.806603+0000","last_undegraded":"2023-10-07T23:21:25.806603+0000","last_fullsized":"2023-10-07T23:21:25.806603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816488+0000","last_change":"2023-10-07T23:21:25.809325+0000","last_active":"2023-10-07T23:21:25.816488+0000","last_peered":"2023-10-07T23:21:25.816488+0000","last_clean":"2023-10-07T23:21:25.816488+0000","last_became_active":"2023-10-07T23:21:25.808944+0000","last_became_peered":"2023-10-07T23:21:25.808944+0000","last_unstale":"2023-10-07T23:21:25.816488+0000","last_undegraded":"2023-10-07T23:21:25.816488+0000","last_fullsized":"2023-10-07T23:21:25.816488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:50.633181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822948+0000","last_change":"2023-10-07T23:21:25.815656+0000","last_active":"2023-10-07T23:21:25.822948+0000","last_peered":"2023-10-07T23:21:25.822948+0000","last_clean":"2023-10-07T23:21:25.822948+0000","last_became_active":"2023-10-07T23:21:25.815290+0000","last_became_peered":"2023-10-07T23:21:25.815290+0000","last_unstale":"2023-10-07T23:21:25.822948+0000","last_undegraded":"2023-10-07T23:21:25.822948+0000","last_fullsized":"2023-10-07T23:21:25.822948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831315+0000","last_change":"2023-10-07T23:21:25.821314+0000","last_active":"2023-10-07T23:21:25.831315+0000","last_peered":"2023-10-07T23:21:25.831315+0000","last_clean":"2023-10-07T23:21:25.831315+0000","last_became_active":"2023-10-07T23:21:25.820716+0000","last_became_peered":"2023-10-07T23:21:25.820716+0000","last_unstale":"2023-10-07T23:21:25.831315+0000","last_undegraded":"2023-10-07T23:21:25.831315+0000","last_fullsized":"2023-10-07T23:21:25.831315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820420+0000","last_change":"2023-10-07T23:21:25.815145+0000","last_active":"2023-10-07T23:21:25.820420+0000","last_peered":"2023-10-07T23:21:25.820420+0000","last_clean":"2023-10-07T23:21:25.820420+0000","last_became_active":"2023-10-07T23:21:25.814843+0000","last_became_peered":"2023-10-07T23:21:25.814843+0000","last_unstale":"2023-10-07T23:21:25.820420+0000","last_undegraded":"2023-10-07T23:21:25.820420+0000","last_fullsized":"2023-10-07T23:21:25.820420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:37.712425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824534+0000","last_change":"2023-10-07T23:21:25.822953+0000","last_active":"2023-10-07T23:21:25.824534+0000","last_peered":"2023-10-07T23:21:25.824534+0000","last_clean":"2023-10-07T23:21:25.824534+0000","last_became_active":"2023-10-07T23:21:25.822567+0000","last_became_peered":"2023-10-07T23:21:25.822567+0000","last_unstale":"2023-10-07T23:21:25.824534+0000","last_undegraded":"2023-10-07T23:21:25.824534+0000","last_fullsized":"2023-10-07T23:21:25.824534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.818161+0000","last_change":"2023-10-07T23:21:25.810275+0000","last_active":"2023-10-07T23:21:25.818161+0000","last_peered":"2023-10-07T23:21:25.818161+0000","last_clean":"2023-10-07T23:21:25.818161+0000","last_became_active":"2023-10-07T23:21:25.808608+0000","last_became_peered":"2023-10-07T23:21:25.808608+0000","last_unstale":"2023-10-07T23:21:25.818161+0000","last_undegraded":"2023-10-07T23:21:25.818161+0000","last_fullsized":"2023-10-07T23:21:25.818161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815069+0000","last_change":"2023-10-07T23:21:25.810226+0000","last_active":"2023-10-07T23:21:25.815069+0000","last_peered":"2023-10-07T23:21:25.815069+0000","last_clean":"2023-10-07T23:21:25.815069+0000","last_became_active":"2023-10-07T23:21:25.809894+0000","last_became_peered":"2023-10-07T23:21:25.809894+0000","last_unstale":"2023-10-07T23:21:25.815069+0000","last_undegraded":"2023-10-07T23:21:25.815069+0000","last_fullsized":"2023-10-07T23:21:25.815069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820902+0000","last_change":"2023-10-07T23:21:25.811504+0000","last_active":"2023-10-07T23:21:25.820902+0000","last_peered":"2023-10-07T23:21:25.820902+0000","last_clean":"2023-10-07T23:21:25.820902+0000","last_became_active":"2023-10-07T23:21:25.811171+0000","last_became_peered":"2023-10-07T23:21:25.811171+0000","last_unstale":"2023-10-07T23:21:25.820902+0000","last_undegraded":"2023-10-07T23:21:25.820902+0000","last_fullsized":"2023-10-07T23:21:25.820902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813279+0000","last_change":"2023-10-07T23:21:25.811409+0000","last_active":"2023-10-07T23:21:25.813279+0000","last_peered":"2023-10-07T23:21:25.813279+0000","last_clean":"2023-10-07T23:21:25.813279+0000","last_became_active":"2023-10-07T23:21:25.811110+0000","last_became_peered":"2023-10-07T23:21:25.811110+0000","last_unstale":"2023-10-07T23:21:25.813279+0000","last_undegraded":"2023-10-07T23:21:25.813279+0000","last_fullsized":"2023-10-07T23:21:25.813279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831600+0000","last_change":"2023-10-07T23:21:25.823226+0000","last_active":"2023-10-07T23:21:25.831600+0000","last_peered":"2023-10-07T23:21:25.831600+0000","last_clean":"2023-10-07T23:21:25.831600+0000","last_became_active":"2023-10-07T23:21:25.822718+0000","last_became_peered":"2023-10-07T23:21:25.822718+0000","last_unstale":"2023-10-07T23:21:25.831600+0000","last_undegraded":"2023-10-07T23:21:25.831600+0000","last_fullsized":"2023-10-07T23:21:25.831600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816264+0000","last_change":"2023-10-07T23:21:25.808833+0000","last_active":"2023-10-07T23:21:25.816264+0000","last_peered":"2023-10-07T23:21:25.816264+0000","last_clean":"2023-10-07T23:21:25.816264+0000","last_became_active":"2023-10-07T23:21:25.808473+0000","last_became_peered":"2023-10-07T23:21:25.808473+0000","last_unstale":"2023-10-07T23:21:25.816264+0000","last_undegraded":"2023-10-07T23:21:25.816264+0000","last_fullsized":"2023-10-07T23:21:25.816264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822506+0000","last_change":"2023-10-07T23:21:25.815523+0000","last_active":"2023-10-07T23:21:25.822506+0000","last_peered":"2023-10-07T23:21:25.822506+0000","last_clean":"2023-10-07T23:21:25.822506+0000","last_became_active":"2023-10-07T23:21:25.814775+0000","last_became_peered":"2023-10-07T23:21:25.814775+0000","last_unstale":"2023-10-07T23:21:25.822506+0000","last_undegraded":"2023-10-07T23:21:25.822506+0000","last_fullsized":"2023-10-07T23:21:25.822506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823146+0000","last_change":"2023-10-07T23:21:25.818446+0000","last_active":"2023-10-07T23:21:25.823146+0000","last_peered":"2023-10-07T23:21:25.823146+0000","last_clean":"2023-10-07T23:21:25.823146+0000","last_became_active":"2023-10-07T23:21:25.818155+0000","last_became_peered":"2023-10-07T23:21:25.818155+0000","last_unstale":"2023-10-07T23:21:25.823146+0000","last_undegraded":"2023-10-07T23:21:25.823146+0000","last_fullsized":"2023-10-07T23:21:25.823146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822397+0000","last_change":"2023-10-07T23:21:25.815227+0000","last_active":"2023-10-07T23:21:25.822397+0000","last_peered":"2023-10-07T23:21:25.822397+0000","last_clean":"2023-10-07T23:21:25.822397+0000","last_became_active":"2023-10-07T23:21:25.814723+0000","last_became_peered":"2023-10-07T23:21:25.814723+0000","last_unstale":"2023-10-07T23:21:25.822397+0000","last_undegraded":"2023-10-07T23:21:25.822397+0000","last_fullsized":"2023-10-07T23:21:25.822397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810308+0000","last_change":"2023-10-07T23:21:25.808952+0000","last_active":"2023-10-07T23:21:25.810308+0000","last_peered":"2023-10-07T23:21:25.810308+0000","last_clean":"2023-10-07T23:21:25.810308+0000","last_became_active":"2023-10-07T23:21:25.808651+0000","last_became_peered":"2023-10-07T23:21:25.808651+0000","last_unstale":"2023-10-07T23:21:25.810308+0000","last_undegraded":"2023-10-07T23:21:25.810308+0000","last_fullsized":"2023-10-07T23:21:25.810308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820498+0000","last_change":"2023-10-07T23:21:25.816989+0000","last_active":"2023-10-07T23:21:25.820498+0000","last_peered":"2023-10-07T23:21:25.820498+0000","last_clean":"2023-10-07T23:21:25.820498+0000","last_became_active":"2023-10-07T23:21:25.816637+0000","last_became_peered":"2023-10-07T23:21:25.816637+0000","last_unstale":"2023-10-07T23:21:25.820498+0000","last_undegraded":"2023-10-07T23:21:25.820498+0000","last_fullsized":"2023-10-07T23:21:25.820498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:34.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824888+0000","last_change":"2023-10-07T23:21:25.817393+0000","last_active":"2023-10-07T23:21:25.824888+0000","last_peered":"2023-10-07T23:21:25.824888+0000","last_clean":"2023-10-07T23:21:25.824888+0000","last_became_active":"2023-10-07T23:21:25.816812+0000","last_became_peered":"2023-10-07T23:21:25.816812+0000","last_unstale":"2023-10-07T23:21:25.824888+0000","last_undegraded":"2023-10-07T23:21:25.824888+0000","last_fullsized":"2023-10-07T23:21:25.824888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831230+0000","last_change":"2023-10-07T23:21:25.820991+0000","last_active":"2023-10-07T23:21:25.831230+0000","last_peered":"2023-10-07T23:21:25.831230+0000","last_clean":"2023-10-07T23:21:25.831230+0000","last_became_active":"2023-10-07T23:21:25.820689+0000","last_became_peered":"2023-10-07T23:21:25.820689+0000","last_unstale":"2023-10-07T23:21:25.831230+0000","last_undegraded":"2023-10-07T23:21:25.831230+0000","last_fullsized":"2023-10-07T23:21:25.831230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822891+0000","last_change":"2023-10-07T23:21:25.818034+0000","last_active":"2023-10-07T23:21:25.822891+0000","last_peered":"2023-10-07T23:21:25.822891+0000","last_clean":"2023-10-07T23:21:25.822891+0000","last_became_active":"2023-10-07T23:21:25.817671+0000","last_became_peered":"2023-10-07T23:21:25.817671+0000","last_unstale":"2023-10-07T23:21:25.822891+0000","last_undegraded":"2023-10-07T23:21:25.822891+0000","last_fullsized":"2023-10-07T23:21:25.822891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816324+0000","last_change":"2023-10-07T23:21:25.814531+0000","last_active":"2023-10-07T23:21:25.816324+0000","last_peered":"2023-10-07T23:21:25.816324+0000","last_clean":"2023-10-07T23:21:25.816324+0000","last_became_active":"2023-10-07T23:21:25.814176+0000","last_became_peered":"2023-10-07T23:21:25.814176+0000","last_unstale":"2023-10-07T23:21:25.816324+0000","last_undegraded":"2023-10-07T23:21:25.816324+0000","last_fullsized":"2023-10-07T23:21:25.816324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:03.964344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819009+0000","last_change":"2023-10-07T23:21:25.815377+0000","last_active":"2023-10-07T23:21:25.819009+0000","last_peered":"2023-10-07T23:21:25.819009+0000","last_clean":"2023-10-07T23:21:25.819009+0000","last_became_active":"2023-10-07T23:21:25.815081+0000","last_became_peered":"2023-10-07T23:21:25.815081+0000","last_unstale":"2023-10-07T23:21:25.819009+0000","last_undegraded":"2023-10-07T23:21:25.819009+0000","last_fullsized":"2023-10-07T23:21:25.819009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816446+0000","last_change":"2023-10-07T23:21:25.809589+0000","last_active":"2023-10-07T23:21:25.816446+0000","last_peered":"2023-10-07T23:21:25.816446+0000","last_clean":"2023-10-07T23:21:25.816446+0000","last_became_active":"2023-10-07T23:21:25.809194+0000","last_became_peered":"2023-10-07T23:21:25.809194+0000","last_unstale":"2023-10-07T23:21:25.816446+0000","last_undegraded":"2023-10-07T23:21:25.816446+0000","last_fullsized":"2023-10-07T23:21:25.816446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:35.323397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.959548+0000","last_change":"2023-10-07T23:21:24.961412+0000","last_active":"2023-10-07T23:21:25.959548+0000","last_peered":"2023-10-07T23:21:25.959548+0000","last_clean":"2023-10-07T23:21:25.959548+0000","last_became_active":"2023-10-07T23:21:24.961027+0000","last_became_peered":"2023-10-07T23:21:24.961027+0000","last_unstale":"2023-10-07T23:21:25.959548+0000","last_undegraded":"2023-10-07T23:21:25.959548+0000","last_fullsized":"2023-10-07T23:21:25.959548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:13.180905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831277+0000","last_change":"2023-10-07T23:21:25.821055+0000","last_active":"2023-10-07T23:21:25.831277+0000","last_peered":"2023-10-07T23:21:25.831277+0000","last_clean":"2023-10-07T23:21:25.831277+0000","last_became_active":"2023-10-07T23:21:25.820736+0000","last_became_peered":"2023-10-07T23:21:25.820736+0000","last_unstale":"2023-10-07T23:21:25.831277+0000","last_undegraded":"2023-10-07T23:21:25.831277+0000","last_fullsized":"2023-10-07T23:21:25.831277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.952918+0000","last_change":"2023-10-07T23:21:24.946739+0000","last_active":"2023-10-07T23:21:25.952918+0000","last_peered":"2023-10-07T23:21:25.952918+0000","last_clean":"2023-10-07T23:21:25.952918+0000","last_became_active":"2023-10-07T23:21:24.946320+0000","last_became_peered":"2023-10-07T23:21:24.946320+0000","last_unstale":"2023-10-07T23:21:25.952918+0000","last_undegraded":"2023-10-07T23:21:25.952918+0000","last_fullsized":"2023-10-07T23:21:25.952918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810406+0000","last_change":"2023-10-07T23:21:24.955775+0000","last_active":"2023-10-07T23:21:25.810406+0000","last_peered":"2023-10-07T23:21:25.810406+0000","last_clean":"2023-10-07T23:21:25.810406+0000","last_became_active":"2023-10-07T23:21:24.955487+0000","last_became_peered":"2023-10-07T23:21:24.955487+0000","last_unstale":"2023-10-07T23:21:25.810406+0000","last_undegraded":"2023-10-07T23:21:25.810406+0000","last_fullsized":"2023-10-07T23:21:25.810406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825497+0000","last_change":"2023-10-07T23:21:25.822494+0000","last_active":"2023-10-07T23:21:25.825497+0000","last_peered":"2023-10-07T23:21:25.825497+0000","last_clean":"2023-10-07T23:21:25.825497+0000","last_became_active":"2023-10-07T23:21:25.821860+0000","last_became_peered":"2023-10-07T23:21:25.821860+0000","last_unstale":"2023-10-07T23:21:25.825497+0000","last_undegraded":"2023-10-07T23:21:25.825497+0000","last_fullsized":"2023-10-07T23:21:25.825497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.796432+0000","last_change":"2023-10-07T23:21:24.949505+0000","last_active":"2023-10-07T23:21:25.796432+0000","last_peered":"2023-10-07T23:21:25.796432+0000","last_clean":"2023-10-07T23:21:25.796432+0000","last_became_active":"2023-10-07T23:21:24.949195+0000","last_became_peered":"2023-10-07T23:21:24.949195+0000","last_unstale":"2023-10-07T23:21:25.796432+0000","last_undegraded":"2023-10-07T23:21:25.796432+0000","last_fullsized":"2023-10-07T23:21:25.796432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820474+0000","last_change":"2023-10-07T23:21:25.816941+0000","last_active":"2023-10-07T23:21:25.820474+0000","last_peered":"2023-10-07T23:21:25.820474+0000","last_clean":"2023-10-07T23:21:25.820474+0000","last_became_active":"2023-10-07T23:21:25.816658+0000","last_became_peered":"2023-10-07T23:21:25.816658+0000","last_unstale":"2023-10-07T23:21:25.820474+0000","last_undegraded":"2023-10-07T23:21:25.820474+0000","last_fullsized":"2023-10-07T23:21:25.820474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:20.455188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794906+0000","last_change":"2023-10-07T23:21:24.953996+0000","last_active":"2023-10-07T23:21:25.794906+0000","last_peered":"2023-10-07T23:21:25.794906+0000","last_clean":"2023-10-07T23:21:25.794906+0000","last_became_active":"2023-10-07T23:21:24.953684+0000","last_became_peered":"2023-10-07T23:21:24.953684+0000","last_unstale":"2023-10-07T23:21:25.794906+0000","last_undegraded":"2023-10-07T23:21:25.794906+0000","last_fullsized":"2023-10-07T23:21:25.794906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816295+0000","last_change":"2023-10-07T23:21:25.809600+0000","last_active":"2023-10-07T23:21:25.816295+0000","last_peered":"2023-10-07T23:21:25.816295+0000","last_clean":"2023-10-07T23:21:25.816295+0000","last_became_active":"2023-10-07T23:21:25.808699+0000","last_became_peered":"2023-10-07T23:21:25.808699+0000","last_unstale":"2023-10-07T23:21:25.816295+0000","last_undegraded":"2023-10-07T23:21:25.816295+0000","last_fullsized":"2023-10-07T23:21:25.816295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.793286+0000","last_change":"2023-10-07T23:21:24.952022+0000","last_active":"2023-10-07T23:21:25.793286+0000","last_peered":"2023-10-07T23:21:25.793286+0000","last_clean":"2023-10-07T23:21:25.793286+0000","last_became_active":"2023-10-07T23:21:24.951291+0000","last_became_peered":"2023-10-07T23:21:24.951291+0000","last_unstale":"2023-10-07T23:21:25.793286+0000","last_undegraded":"2023-10-07T23:21:25.793286+0000","last_fullsized":"2023-10-07T23:21:25.793286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823818+0000","last_change":"2023-10-07T23:21:25.816148+0000","last_active":"2023-10-07T23:21:25.823818+0000","last_peered":"2023-10-07T23:21:25.823818+0000","last_clean":"2023-10-07T23:21:25.823818+0000","last_became_active":"2023-10-07T23:21:25.815705+0000","last_became_peered":"2023-10-07T23:21:25.815705+0000","last_unstale":"2023-10-07T23:21:25.823818+0000","last_undegraded":"2023-10-07T23:21:25.823818+0000","last_fullsized":"2023-10-07T23:21:25.823818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:23.605003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954881+0000","last_change":"2023-10-07T23:21:24.954956+0000","last_active":"2023-10-07T23:21:25.954881+0000","last_peered":"2023-10-07T23:21:25.954881+0000","last_clean":"2023-10-07T23:21:25.954881+0000","last_became_active":"2023-10-07T23:21:24.954596+0000","last_became_peered":"2023-10-07T23:21:24.954596+0000","last_unstale":"2023-10-07T23:21:25.954881+0000","last_undegraded":"2023-10-07T23:21:25.954881+0000","last_fullsized":"2023-10-07T23:21:25.954881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:02.562726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816506+0000","last_change":"2023-10-07T23:21:25.809979+0000","last_active":"2023-10-07T23:21:25.816506+0000","last_peered":"2023-10-07T23:21:25.816506+0000","last_clean":"2023-10-07T23:21:25.816506+0000","last_became_active":"2023-10-07T23:21:25.809594+0000","last_became_peered":"2023-10-07T23:21:25.809594+0000","last_unstale":"2023-10-07T23:21:25.816506+0000","last_undegraded":"2023-10-07T23:21:25.816506+0000","last_fullsized":"2023-10-07T23:21:25.816506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:28.775950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.966427+0000","last_change":"2023-10-07T23:21:24.954704+0000","last_active":"2023-10-07T23:21:25.966427+0000","last_peered":"2023-10-07T23:21:25.966427+0000","last_clean":"2023-10-07T23:21:25.966427+0000","last_became_active":"2023-10-07T23:21:24.954416+0000","last_became_peered":"2023-10-07T23:21:24.954416+0000","last_unstale":"2023-10-07T23:21:25.966427+0000","last_undegraded":"2023-10-07T23:21:25.966427+0000","last_fullsized":"2023-10-07T23:21:25.966427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824095+0000","last_change":"2023-10-07T23:21:25.817538+0000","last_active":"2023-10-07T23:21:25.824095+0000","last_peered":"2023-10-07T23:21:25.824095+0000","last_clean":"2023-10-07T23:21:25.824095+0000","last_became_active":"2023-10-07T23:21:25.817209+0000","last_became_peered":"2023-10-07T23:21:25.817209+0000","last_unstale":"2023-10-07T23:21:25.824095+0000","last_undegraded":"2023-10-07T23:21:25.824095+0000","last_fullsized":"2023-10-07T23:21:25.824095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964849+0000","last_change":"2023-10-07T23:21:24.957350+0000","last_active":"2023-10-07T23:21:25.964849+0000","last_peered":"2023-10-07T23:21:25.964849+0000","last_clean":"2023-10-07T23:21:25.964849+0000","last_became_active":"2023-10-07T23:21:24.957009+0000","last_became_peered":"2023-10-07T23:21:24.957009+0000","last_unstale":"2023-10-07T23:21:25.964849+0000","last_undegraded":"2023-10-07T23:21:25.964849+0000","last_fullsized":"2023-10-07T23:21:25.964849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:58.493058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823798+0000","last_change":"2023-10-07T23:21:25.814807+0000","last_active":"2023-10-07T23:21:25.823798+0000","last_peered":"2023-10-07T23:21:25.823798+0000","last_clean":"2023-10-07T23:21:25.823798+0000","last_became_active":"2023-10-07T23:21:25.813589+0000","last_became_peered":"2023-10-07T23:21:25.813589+0000","last_unstale":"2023-10-07T23:21:25.823798+0000","last_undegraded":"2023-10-07T23:21:25.823798+0000","last_fullsized":"2023-10-07T23:21:25.823798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797030+0000","last_change":"2023-10-07T23:21:24.957704+0000","last_active":"2023-10-07T23:21:25.797030+0000","last_peered":"2023-10-07T23:21:25.797030+0000","last_clean":"2023-10-07T23:21:25.797030+0000","last_became_active":"2023-10-07T23:21:24.957447+0000","last_became_peered":"2023-10-07T23:21:24.957447+0000","last_unstale":"2023-10-07T23:21:25.797030+0000","last_undegraded":"2023-10-07T23:21:25.797030+0000","last_fullsized":"2023-10-07T23:21:25.797030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824121+0000","last_change":"2023-10-07T23:21:25.817566+0000","last_active":"2023-10-07T23:21:25.824121+0000","last_peered":"2023-10-07T23:21:25.824121+0000","last_clean":"2023-10-07T23:21:25.824121+0000","last_became_active":"2023-10-07T23:21:25.816898+0000","last_became_peered":"2023-10-07T23:21:25.816898+0000","last_unstale":"2023-10-07T23:21:25.824121+0000","last_undegraded":"2023-10-07T23:21:25.824121+0000","last_fullsized":"2023-10-07T23:21:25.824121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794453+0000","last_change":"2023-10-07T23:21:24.948051+0000","last_active":"2023-10-07T23:21:25.794453+0000","last_peered":"2023-10-07T23:21:25.794453+0000","last_clean":"2023-10-07T23:21:25.794453+0000","last_became_active":"2023-10-07T23:21:24.947742+0000","last_became_peered":"2023-10-07T23:21:24.947742+0000","last_unstale":"2023-10-07T23:21:25.794453+0000","last_undegraded":"2023-10-07T23:21:25.794453+0000","last_fullsized":"2023-10-07T23:21:25.794453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819027+0000","last_change":"2023-10-07T23:21:25.815665+0000","last_active":"2023-10-07T23:21:25.819027+0000","last_peered":"2023-10-07T23:21:25.819027+0000","last_clean":"2023-10-07T23:21:25.819027+0000","last_became_active":"2023-10-07T23:21:25.815286+0000","last_became_peered":"2023-10-07T23:21:25.815286+0000","last_unstale":"2023-10-07T23:21:25.819027+0000","last_undegraded":"2023-10-07T23:21:25.819027+0000","last_fullsized":"2023-10-07T23:21:25.819027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.960851+0000","last_change":"2023-10-07T23:21:24.952011+0000","last_active":"2023-10-07T23:21:25.960851+0000","last_peered":"2023-10-07T23:21:25.960851+0000","last_clean":"2023-10-07T23:21:25.960851+0000","last_became_active":"2023-10-07T23:21:24.951691+0000","last_became_peered":"2023-10-07T23:21:24.951691+0000","last_unstale":"2023-10-07T23:21:25.960851+0000","last_undegraded":"2023-10-07T23:21:25.960851+0000","last_fullsized":"2023-10-07T23:21:25.960851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823976+0000","last_change":"2023-10-07T23:21:25.817054+0000","last_active":"2023-10-07T23:21:25.823976+0000","last_peered":"2023-10-07T23:21:25.823976+0000","last_clean":"2023-10-07T23:21:25.823976+0000","last_became_active":"2023-10-07T23:21:25.816667+0000","last_became_peered":"2023-10-07T23:21:25.816667+0000","last_unstale":"2023-10-07T23:21:25.823976+0000","last_undegraded":"2023-10-07T23:21:25.823976+0000","last_fullsized":"2023-10-07T23:21:25.823976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.961478+0000","last_change":"2023-10-07T23:21:24.958951+0000","last_active":"2023-10-07T23:21:25.961478+0000","last_peered":"2023-10-07T23:21:25.961478+0000","last_clean":"2023-10-07T23:21:25.961478+0000","last_became_active":"2023-10-07T23:21:24.958632+0000","last_became_peered":"2023-10-07T23:21:24.958632+0000","last_unstale":"2023-10-07T23:21:25.961478+0000","last_undegraded":"2023-10-07T23:21:25.961478+0000","last_fullsized":"2023-10-07T23:21:25.961478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:29.730287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823870+0000","last_change":"2023-10-07T23:21:25.816914+0000","last_active":"2023-10-07T23:21:25.823870+0000","last_peered":"2023-10-07T23:21:25.823870+0000","last_clean":"2023-10-07T23:21:25.823870+0000","last_became_active":"2023-10-07T23:21:25.816515+0000","last_became_peered":"2023-10-07T23:21:25.816515+0000","last_unstale":"2023-10-07T23:21:25.823870+0000","last_undegraded":"2023-10-07T23:21:25.823870+0000","last_fullsized":"2023-10-07T23:21:25.823870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.955373+0000","last_change":"2023-10-07T23:21:24.956912+0000","last_active":"2023-10-07T23:21:25.955373+0000","last_peered":"2023-10-07T23:21:25.955373+0000","last_clean":"2023-10-07T23:21:25.955373+0000","last_became_active":"2023-10-07T23:21:24.956588+0000","last_became_peered":"2023-10-07T23:21:24.956588+0000","last_unstale":"2023-10-07T23:21:25.955373+0000","last_undegraded":"2023-10-07T23:21:25.955373+0000","last_fullsized":"2023-10-07T23:21:25.955373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824203+0000","last_change":"2023-10-07T23:21:25.818653+0000","last_active":"2023-10-07T23:21:25.824203+0000","last_peered":"2023-10-07T23:21:25.824203+0000","last_clean":"2023-10-07T23:21:25.824203+0000","last_became_active":"2023-10-07T23:21:25.817833+0000","last_became_peered":"2023-10-07T23:21:25.817833+0000","last_unstale":"2023-10-07T23:21:25.824203+0000","last_undegraded":"2023-10-07T23:21:25.824203+0000","last_fullsized":"2023-10-07T23:21:25.824203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954656+0000","last_change":"2023-10-07T23:21:24.948082+0000","last_active":"2023-10-07T23:21:25.954656+0000","last_peered":"2023-10-07T23:21:25.954656+0000","last_clean":"2023-10-07T23:21:25.954656+0000","last_became_active":"2023-10-07T23:21:24.947759+0000","last_became_peered":"2023-10-07T23:21:24.947759+0000","last_unstale":"2023-10-07T23:21:25.954656+0000","last_undegraded":"2023-10-07T23:21:25.954656+0000","last_fullsized":"2023-10-07T23:21:25.954656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:11.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822945+0000","last_change":"2023-10-07T23:21:25.817833+0000","last_active":"2023-10-07T23:21:25.822945+0000","last_peered":"2023-10-07T23:21:25.822945+0000","last_clean":"2023-10-07T23:21:25.822945+0000","last_became_active":"2023-10-07T23:21:25.817547+0000","last_became_peered":"2023-10-07T23:21:25.817547+0000","last_unstale":"2023-10-07T23:21:25.822945+0000","last_undegraded":"2023-10-07T23:21:25.822945+0000","last_fullsized":"2023-10-07T23:21:25.822945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.807532+0000","last_change":"2023-10-07T23:21:24.956344+0000","last_active":"2023-10-07T23:21:25.807532+0000","last_peered":"2023-10-07T23:21:25.807532+0000","last_clean":"2023-10-07T23:21:25.807532+0000","last_became_active":"2023-10-07T23:21:24.955974+0000","last_became_peered":"2023-10-07T23:21:24.955974+0000","last_unstale":"2023-10-07T23:21:25.807532+0000","last_undegraded":"2023-10-07T23:21:25.807532+0000","last_fullsized":"2023-10-07T23:21:25.807532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.816194+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.813965+0000","last_became_peered":"2023-10-07T23:21:25.813965+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802932+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:21:25.802932+0000","last_peered":"2023-10-07T23:21:25.802932+0000","last_clean":"2023-10-07T23:21:25.802932+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:21:25.802932+0000","last_undegraded":"2023-10-07T23:21:25.802932+0000","last_fullsized":"2023-10-07T23:21:25.802932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.792857+0000","last_change":"2023-10-07T23:21:24.953534+0000","last_active":"2023-10-07T23:21:25.792857+0000","last_peered":"2023-10-07T23:21:25.792857+0000","last_clean":"2023-10-07T23:21:25.792857+0000","last_became_active":"2023-10-07T23:21:24.953202+0000","last_became_peered":"2023-10-07T23:21:24.953202+0000","last_unstale":"2023-10-07T23:21:25.792857+0000","last_undegraded":"2023-10-07T23:21:25.792857+0000","last_fullsized":"2023-10-07T23:21:25.792857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824763+0000","last_change":"2023-10-07T23:21:25.816874+0000","last_active":"2023-10-07T23:21:25.824763+0000","last_peered":"2023-10-07T23:21:25.824763+0000","last_clean":"2023-10-07T23:21:25.824763+0000","last_became_active":"2023-10-07T23:21:25.816098+0000","last_became_peered":"2023-10-07T23:21:25.816098+0000","last_unstale":"2023-10-07T23:21:25.824763+0000","last_undegraded":"2023-10-07T23:21:25.824763+0000","last_fullsized":"2023-10-07T23:21:25.824763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802840+0000","last_change":"2023-10-07T23:21:24.962711+0000","last_active":"2023-10-07T23:21:25.802840+0000","last_peered":"2023-10-07T23:21:25.802840+0000","last_clean":"2023-10-07T23:21:25.802840+0000","last_became_active":"2023-10-07T23:21:24.962404+0000","last_became_peered":"2023-10-07T23:21:24.962404+0000","last_unstale":"2023-10-07T23:21:25.802840+0000","last_undegraded":"2023-10-07T23:21:25.802840+0000","last_fullsized":"2023-10-07T23:21:25.802840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:02.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819414+0000","last_change":"2023-10-07T23:21:25.809376+0000","last_active":"2023-10-07T23:21:25.819414+0000","last_peered":"2023-10-07T23:21:25.819414+0000","last_clean":"2023-10-07T23:21:25.819414+0000","last_became_active":"2023-10-07T23:21:25.809055+0000","last_became_peered":"2023-10-07T23:21:25.809055+0000","last_unstale":"2023-10-07T23:21:25.819414+0000","last_undegraded":"2023-10-07T23:21:25.819414+0000","last_fullsized":"2023-10-07T23:21:25.819414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806772+0000","last_change":"2023-10-07T23:21:24.960450+0000","last_active":"2023-10-07T23:21:25.806772+0000","last_peered":"2023-10-07T23:21:25.806772+0000","last_clean":"2023-10-07T23:21:25.806772+0000","last_became_active":"2023-10-07T23:21:24.960068+0000","last_became_peered":"2023-10-07T23:21:24.960068+0000","last_unstale":"2023-10-07T23:21:25.806772+0000","last_undegraded":"2023-10-07T23:21:25.806772+0000","last_fullsized":"2023-10-07T23:21:25.806772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820366+0000","last_change":"2023-10-07T23:21:25.815004+0000","last_active":"2023-10-07T23:21:25.820366+0000","last_peered":"2023-10-07T23:21:25.820366+0000","last_clean":"2023-10-07T23:21:25.820366+0000","last_became_active":"2023-10-07T23:21:25.814688+0000","last_became_peered":"2023-10-07T23:21:25.814688+0000","last_unstale":"2023-10-07T23:21:25.820366+0000","last_undegraded":"2023-10-07T23:21:25.820366+0000","last_fullsized":"2023-10-07T23:21:25.820366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:46.190235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797147+0000","last_change":"2023-10-07T23:21:24.952189+0000","last_active":"2023-10-07T23:21:25.797147+0000","last_peered":"2023-10-07T23:21:25.797147+0000","last_clean":"2023-10-07T23:21:25.797147+0000","last_became_active":"2023-10-07T23:21:24.951731+0000","last_became_peered":"2023-10-07T23:21:24.951731+0000","last_unstale":"2023-10-07T23:21:25.797147+0000","last_undegraded":"2023-10-07T23:21:25.797147+0000","last_fullsized":"2023-10-07T23:21:25.797147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823922+0000","last_change":"2023-10-07T23:21:25.822875+0000","last_active":"2023-10-07T23:21:25.823922+0000","last_peered":"2023-10-07T23:21:25.823922+0000","last_clean":"2023-10-07T23:21:25.823922+0000","last_became_active":"2023-10-07T23:21:25.822536+0000","last_became_peered":"2023-10-07T23:21:25.822536+0000","last_unstale":"2023-10-07T23:21:25.823922+0000","last_undegraded":"2023-10-07T23:21:25.823922+0000","last_fullsized":"2023-10-07T23:21:25.823922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:47.179786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.800662+0000","last_change":"2023-10-07T23:21:24.952562+0000","last_active":"2023-10-07T23:21:25.800662+0000","last_peered":"2023-10-07T23:21:25.800662+0000","last_clean":"2023-10-07T23:21:25.800662+0000","last_became_active":"2023-10-07T23:21:24.952071+0000","last_became_peered":"2023-10-07T23:21:24.952071+0000","last_unstale":"2023-10-07T23:21:25.800662+0000","last_undegraded":"2023-10-07T23:21:25.800662+0000","last_fullsized":"2023-10-07T23:21:25.800662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:17.603304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822867+0000","last_change":"2023-10-07T23:21:25.818214+0000","last_active":"2023-10-07T23:21:25.822867+0000","last_peered":"2023-10-07T23:21:25.822867+0000","last_clean":"2023-10-07T23:21:25.822867+0000","last_became_active":"2023-10-07T23:21:25.817888+0000","last_became_peered":"2023-10-07T23:21:25.817888+0000","last_unstale":"2023-10-07T23:21:25.822867+0000","last_undegraded":"2023-10-07T23:21:25.822867+0000","last_fullsized":"2023-10-07T23:21:25.822867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.804473+0000","last_change":"2023-10-07T23:21:24.962199+0000","last_active":"2023-10-07T23:21:25.804473+0000","last_peered":"2023-10-07T23:21:25.804473+0000","last_clean":"2023-10-07T23:21:25.804473+0000","last_became_active":"2023-10-07T23:21:24.961778+0000","last_became_peered":"2023-10-07T23:21:24.961778+0000","last_unstale":"2023-10-07T23:21:25.804473+0000","last_undegraded":"2023-10-07T23:21:25.804473+0000","last_fullsized":"2023-10-07T23:21:25.804473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820428+0000","last_change":"2023-10-07T23:21:25.815292+0000","last_active":"2023-10-07T23:21:25.820428+0000","last_peered":"2023-10-07T23:21:25.820428+0000","last_clean":"2023-10-07T23:21:25.820428+0000","last_became_active":"2023-10-07T23:21:25.814294+0000","last_became_peered":"2023-10-07T23:21:25.814294+0000","last_unstale":"2023-10-07T23:21:25.820428+0000","last_undegraded":"2023-10-07T23:21:25.820428+0000","last_fullsized":"2023-10-07T23:21:25.820428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:41.638201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-07T23:21:27.097216+0000","last_change":"2023-10-07T23:21:24.948131+0000","last_active":"2023-10-07T23:21:27.097216+0000","last_peered":"2023-10-07T23:21:27.097216+0000","last_clean":"2023-10-07T23:21:27.097216+0000","last_became_active":"2023-10-07T23:21:24.947831+0000","last_became_peered":"2023-10-07T23:21:24.947831+0000","last_unstale":"2023-10-07T23:21:27.097216+0000","last_undegraded":"2023-10-07T23:21:27.097216+0000","last_fullsized":"2023-10-07T23:21:27.097216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:23.995859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825605+0000","last_change":"2023-10-07T23:21:25.823117+0000","last_active":"2023-10-07T23:21:25.825605+0000","last_peered":"2023-10-07T23:21:25.825605+0000","last_clean":"2023-10-07T23:21:25.825605+0000","last_became_active":"2023-10-07T23:21:25.822814+0000","last_became_peered":"2023-10-07T23:21:25.822814+0000","last_unstale":"2023-10-07T23:21:25.825605+0000","last_undegraded":"2023-10-07T23:21:25.825605+0000","last_fullsized":"2023-10-07T23:21:25.825605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954544+0000","last_change":"2023-10-07T23:21:24.957129+0000","last_active":"2023-10-07T23:21:25.954544+0000","last_peered":"2023-10-07T23:21:25.954544+0000","last_clean":"2023-10-07T23:21:25.954544+0000","last_became_active":"2023-10-07T23:21:24.956815+0000","last_became_peered":"2023-10-07T23:21:24.956815+0000","last_unstale":"2023-10-07T23:21:25.954544+0000","last_undegraded":"2023-10-07T23:21:25.954544+0000","last_fullsized":"2023-10-07T23:21:25.954544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823723+0000","last_change":"2023-10-07T23:21:25.814482+0000","last_active":"2023-10-07T23:21:25.823723+0000","last_peered":"2023-10-07T23:21:25.823723+0000","last_clean":"2023-10-07T23:21:25.823723+0000","last_became_active":"2023-10-07T23:21:25.814126+0000","last_became_peered":"2023-10-07T23:21:25.814126+0000","last_unstale":"2023-10-07T23:21:25.823723+0000","last_undegraded":"2023-10-07T23:21:25.823723+0000","last_fullsized":"2023-10-07T23:21:25.823723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802684+0000","last_change":"2023-10-07T23:21:24.960263+0000","last_active":"2023-10-07T23:21:25.802684+0000","last_peered":"2023-10-07T23:21:25.802684+0000","last_clean":"2023-10-07T23:21:25.802684+0000","last_became_active":"2023-10-07T23:21:24.959984+0000","last_became_peered":"2023-10-07T23:21:24.959984+0000","last_unstale":"2023-10-07T23:21:25.802684+0000","last_undegraded":"2023-10-07T23:21:25.802684+0000","last_fullsized":"2023-10-07T23:21:25.802684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:34.309999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.821174+0000","last_change":"2023-10-07T23:21:25.819404+0000","last_active":"2023-10-07T23:21:25.821174+0000","last_peered":"2023-10-07T23:21:25.821174+0000","last_clean":"2023-10-07T23:21:25.821174+0000","last_became_active":"2023-10-07T23:21:25.818658+0000","last_became_peered":"2023-10-07T23:21:25.818658+0000","last_unstale":"2023-10-07T23:21:25.821174+0000","last_undegraded":"2023-10-07T23:21:25.821174+0000","last_fullsized":"2023-10-07T23:21:25.821174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.799830+0000","last_change":"2023-10-07T23:21:24.958141+0000","last_active":"2023-10-07T23:21:25.799830+0000","last_peered":"2023-10-07T23:21:25.799830+0000","last_clean":"2023-10-07T23:21:25.799830+0000","last_became_active":"2023-10-07T23:21:24.957791+0000","last_became_peered":"2023-10-07T23:21:24.957791+0000","last_unstale":"2023-10-07T23:21:25.799830+0000","last_undegraded":"2023-10-07T23:21:25.799830+0000","last_fullsized":"2023-10-07T23:21:25.799830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:01.467025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825628+0000","last_change":"2023-10-07T23:21:25.823525+0000","last_active":"2023-10-07T23:21:25.825628+0000","last_peered":"2023-10-07T23:21:25.825628+0000","last_clean":"2023-10-07T23:21:25.825628+0000","last_became_active":"2023-10-07T23:21:25.822193+0000","last_became_peered":"2023-10-07T23:21:25.822193+0000","last_unstale":"2023-10-07T23:21:25.825628+0000","last_undegraded":"2023-10-07T23:21:25.825628+0000","last_fullsized":"2023-10-07T23:21:25.825628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.798501+0000","last_change":"2023-10-07T23:21:24.957341+0000","last_active":"2023-10-07T23:21:25.798501+0000","last_peered":"2023-10-07T23:21:25.798501+0000","last_clean":"2023-10-07T23:21:25.798501+0000","last_became_active":"2023-10-07T23:21:24.957059+0000","last_became_peered":"2023-10-07T23:21:24.957059+0000","last_unstale":"2023-10-07T23:21:25.798501+0000","last_undegraded":"2023-10-07T23:21:25.798501+0000","last_fullsized":"2023-10-07T23:21:25.798501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.702939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825528+0000","last_change":"2023-10-07T23:21:25.822814+0000","last_active":"2023-10-07T23:21:25.825528+0000","last_peered":"2023-10-07T23:21:25.825528+0000","last_clean":"2023-10-07T23:21:25.825528+0000","last_became_active":"2023-10-07T23:21:25.822297+0000","last_became_peered":"2023-10-07T23:21:25.822297+0000","last_unstale":"2023-10-07T23:21:25.825528+0000","last_undegraded":"2023-10-07T23:21:25.825528+0000","last_fullsized":"2023-10-07T23:21:25.825528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954772+0000","last_change":"2023-10-07T23:21:24.961807+0000","last_active":"2023-10-07T23:21:25.954772+0000","last_peered":"2023-10-07T23:21:25.954772+0000","last_clean":"2023-10-07T23:21:25.954772+0000","last_became_active":"2023-10-07T23:21:24.961451+0000","last_became_peered":"2023-10-07T23:21:24.961451+0000","last_unstale":"2023-10-07T23:21:25.954772+0000","last_undegraded":"2023-10-07T23:21:25.954772+0000","last_fullsized":"2023-10-07T23:21:25.954772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813287+0000","last_change":"2023-10-07T23:21:25.811192+0000","last_active":"2023-10-07T23:21:25.813287+0000","last_peered":"2023-10-07T23:21:25.813287+0000","last_clean":"2023-10-07T23:21:25.813287+0000","last_became_active":"2023-10-07T23:21:25.810877+0000","last_became_peered":"2023-10-07T23:21:25.810877+0000","last_unstale":"2023-10-07T23:21:25.813287+0000","last_undegraded":"2023-10-07T23:21:25.813287+0000","last_fullsized":"2023-10-07T23:21:25.813287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816476+0000","last_change":"2023-10-07T23:21:25.811521+0000","last_active":"2023-10-07T23:21:25.816476+0000","last_peered":"2023-10-07T23:21:25.816476+0000","last_clean":"2023-10-07T23:21:25.816476+0000","last_became_active":"2023-10-07T23:21:25.811256+0000","last_became_peered":"2023-10-07T23:21:25.811256+0000","last_unstale":"2023-10-07T23:21:25.816476+0000","last_undegraded":"2023-10-07T23:21:25.816476+0000","last_fullsized":"2023-10-07T23:21:25.816476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954460+0000","last_change":"2023-10-07T23:21:24.962057+0000","last_active":"2023-10-07T23:21:25.954460+0000","last_peered":"2023-10-07T23:21:25.954460+0000","last_clean":"2023-10-07T23:21:25.954460+0000","last_became_active":"2023-10-07T23:21:24.961540+0000","last_became_peered":"2023-10-07T23:21:24.961540+0000","last_unstale":"2023-10-07T23:21:25.954460+0000","last_undegraded":"2023-10-07T23:21:25.954460+0000","last_fullsized":"2023-10-07T23:21:25.954460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.817631+0000","last_change":"2023-10-07T23:21:25.812184+0000","last_active":"2023-10-07T23:21:25.817631+0000","last_peered":"2023-10-07T23:21:25.817631+0000","last_clean":"2023-10-07T23:21:25.817631+0000","last_became_active":"2023-10-07T23:21:25.811844+0000","last_became_peered":"2023-10-07T23:21:25.811844+0000","last_unstale":"2023-10-07T23:21:25.817631+0000","last_undegraded":"2023-10-07T23:21:25.817631+0000","last_fullsized":"2023-10-07T23:21:25.817631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954389+0000","last_change":"2023-10-07T23:21:24.961653+0000","last_active":"2023-10-07T23:21:25.954389+0000","last_peered":"2023-10-07T23:21:25.954389+0000","last_clean":"2023-10-07T23:21:25.954389+0000","last_became_active":"2023-10-07T23:21:24.961362+0000","last_became_peered":"2023-10-07T23:21:24.961362+0000","last_unstale":"2023-10-07T23:21:25.954389+0000","last_undegraded":"2023-10-07T23:21:25.954389+0000","last_fullsized":"2023-10-07T23:21:25.954389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:27.376709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823793+0000","last_change":"2023-10-07T23:21:25.813813+0000","last_active":"2023-10-07T23:21:25.823793+0000","last_peered":"2023-10-07T23:21:25.823793+0000","last_clean":"2023-10-07T23:21:25.823793+0000","last_became_active":"2023-10-07T23:21:25.813500+0000","last_became_peered":"2023-10-07T23:21:25.813500+0000","last_unstale":"2023-10-07T23:21:25.823793+0000","last_undegraded":"2023-10-07T23:21:25.823793+0000","last_fullsized":"2023-10-07T23:21:25.823793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.965038+0000","last_change":"2023-10-07T23:21:24.959465+0000","last_active":"2023-10-07T23:21:25.965038+0000","last_peered":"2023-10-07T23:21:25.965038+0000","last_clean":"2023-10-07T23:21:25.965038+0000","last_became_active":"2023-10-07T23:21:24.957700+0000","last_became_peered":"2023-10-07T23:21:24.957700+0000","last_unstale":"2023-10-07T23:21:25.965038+0000","last_undegraded":"2023-10-07T23:21:25.965038+0000","last_fullsized":"2023-10-07T23:21:25.965038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824672+0000","last_change":"2023-10-07T23:21:25.816098+0000","last_active":"2023-10-07T23:21:25.824672+0000","last_peered":"2023-10-07T23:21:25.824672+0000","last_clean":"2023-10-07T23:21:25.824672+0000","last_became_active":"2023-10-07T23:21:25.815533+0000","last_became_peered":"2023-10-07T23:21:25.815533+0000","last_unstale":"2023-10-07T23:21:25.824672+0000","last_undegraded":"2023-10-07T23:21:25.824672+0000","last_fullsized":"2023-10-07T23:21:25.824672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964792+0000","last_change":"2023-10-07T23:21:24.960874+0000","last_active":"2023-10-07T23:21:25.964792+0000","last_peered":"2023-10-07T23:21:25.964792+0000","last_clean":"2023-10-07T23:21:25.964792+0000","last_became_active":"2023-10-07T23:21:24.960568+0000","last_became_peered":"2023-10-07T23:21:24.960568+0000","last_unstale":"2023-10-07T23:21:25.964792+0000","last_undegraded":"2023-10-07T23:21:25.964792+0000","last_fullsized":"2023-10-07T23:21:25.964792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":671553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":82167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":80757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]}]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301624,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070216,"statfs":{"total":96636764160,"available":96327901184,"internally_reserved":0,"allocated":245760,"data_stored":82167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301612,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070228,"statfs":{"total":96636764160,"available":96327913472,"internally_reserved":0,"allocated":233472,"data_stored":80631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302204,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069636,"statfs":{"total":96636764160,"available":96327307264,"internally_reserved":0,"allocated":839680,"data_stored":671609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":80791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:21:32.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:21:33.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.058+0000 7efc488d8700 1 Processor -- start 2023-10-07T23:21:33.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.058+0000 7efc488d8700 1 -- start start 2023-10-07T23:21:33.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.058+0000 7efc488d8700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc440afc00 0x7efc440affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.058+0000 7efc488d8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.058+0000 7efc488d8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc440b0cf0 con 0x7efc4406c800 2023-10-07T23:21:33.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc440c9c90 con 0x7efc440afc00 2023-10-07T23:21:33.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440c9f70 con 0x7efc440caf20 2023-10-07T23:21:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57130/0 (socket says 172.21.15.57:57130) 2023-10-07T23:21:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 -- 172.21.15.57:0/1165285429 learned_addr learned my addr 172.21.15.57:0/1165285429 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:33.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc42d9d700 1 --2- 172.21.15.57:0/1165285429 >> v2:172.21.15.154:3300/0 conn(0x7efc440afc00 0x7efc440affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 -- 172.21.15.57:0/1165285429 >> v2:172.21.15.154:3300/0 conn(0x7efc440afc00 msgr2=0x7efc440affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 --2- 172.21.15.57:0/1165285429 >> v2:172.21.15.154:3300/0 conn(0x7efc440afc00 0x7efc440affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 -- 172.21.15.57:0/1165285429 >> v1:172.21.15.154:6789/0 conn(0x7efc4406c800 legacy=0x7efc4406cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 -- 172.21.15.57:0/1165285429 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440ca250 con 0x7efc440caf20 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc4259c700 1 --2- 172.21.15.57:0/1165285429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0099b0 tx=0x7efc2c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170a3bd6659468ca server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc41d9b700 1 -- 172.21.15.57:0/1165285429 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c009ea0 con 0x7efc440caf20 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc41d9b700 1 -- 172.21.15.57:0/1165285429 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c00b860 con 0x7efc440caf20 2023-10-07T23:21:33.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc41d9b700 1 -- 172.21.15.57:0/1165285429 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00bc00 con 0x7efc440caf20 2023-10-07T23:21:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- 172.21.15.57:0/1165285429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 msgr2=0x7efc440c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 --2- 172.21.15.57:0/1165285429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0099b0 tx=0x7efc2c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- 172.21.15.57:0/1165285429 shutdown_connections 2023-10-07T23:21:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 --2- 172.21.15.57:0/1165285429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c9340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 --2- 172.21.15.57:0/1165285429 >> v2:172.21.15.154:3300/0 conn(0x7efc440afc00 0x7efc440affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- 172.21.15.57:0/1165285429 >> 172.21.15.57:0/1165285429 conn(0x7efc440692f0 msgr2=0x7efc440696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- 172.21.15.57:0/1165285429 shutdown_connections 2023-10-07T23:21:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.062+0000 7efc488d8700 1 -- 172.21.15.57:0/1165285429 wait complete. 2023-10-07T23:21:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 Processor -- start 2023-10-07T23:21:33.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- start start 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc440afc00 0x7efc440c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc44158ce0 con 0x7efc4406c800 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc440c3db0 con 0x7efc440afc00 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc440c4240 con 0x7efc440caf20 2023-10-07T23:21:33.072 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58646/0 (socket says 172.21.15.57:58646) 2023-10-07T23:21:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 -- 172.21.15.57:0/3687366807 learned_addr learned my addr 172.21.15.57:0/3687366807 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc4259c700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc440afc00 0x7efc440c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc4359e700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 -- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 msgr2=0x7efc440c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 -- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc440afc00 msgr2=0x7efc440c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc440afc00 0x7efc440c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 -- 172.21.15.57:0/3687366807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c0094f0 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc42d9d700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc3400ec80 tx=0x7efc340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400c850 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3400c9b0 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc440c31a0 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400cd50 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.066+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc440c3760 con 0x7efc4406c800 2023-10-07T23:21:33.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.070+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc440711c0 con 0x7efc4406c800 2023-10-07T23:21:33.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.070+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efc34013680 con 0x7efc4406c800 2023-10-07T23:21:33.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.070+0000 7efc3affd700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 0x7efc24062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.070+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc3407d230 con 0x7efc4406c800 2023-10-07T23:21:33.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.074+0000 7efc4259c700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 0x7efc24062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.074+0000 7efc4259c700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 0x7efc24062a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0099b0 tx=0x7efc2c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.078+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efc3404d800 con 0x7efc4406c800 2023-10-07T23:21:33.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.226+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc440b0980 con 0x7efc24060580 2023-10-07T23:21:33.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.246+0000 7efc3affd700 1 -- 172.21.15.57:0/3687366807 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264346 (secure 0 0 0) 0x7efc440b0980 con 0x7efc24060580 2023-10-07T23:21:33.253 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:33.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 msgr2=0x7efc24062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 0x7efc24062a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0099b0 tx=0x7efc2c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:33.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 msgr2=0x7efc441461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc3400ec80 tx=0x7efc340091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 shutdown_connections 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc440caf20 0x7efc440c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc24060580 0x7efc24062a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc440afc00 0x7efc440c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 --2- 172.21.15.57:0/3687366807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc4406c800 0x7efc441461c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 >> 172.21.15.57:0/3687366807 conn(0x7efc440692f0 msgr2=0x7efc4406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 shutdown_connections 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.250+0000 7efc488d8700 1 -- 172.21.15.57:0/3687366807 wait complete. 2023-10-07T23:21:33.257 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:21:33.271 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-07T23:21:33.072655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397736,"kb_used_data":3048,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576984,"statfs":{"total":773094113280,"available":770638831616,"internally_reserved":0,"allocated":3121152,"data_stored":1830396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003106"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824034+0000","last_change":"2023-10-07T23:21:25.817169+0000","last_active":"2023-10-07T23:21:25.824034+0000","last_peered":"2023-10-07T23:21:25.824034+0000","last_clean":"2023-10-07T23:21:25.824034+0000","last_became_active":"2023-10-07T23:21:25.816152+0000","last_became_peered":"2023-10-07T23:21:25.816152+0000","last_unstale":"2023-10-07T23:21:25.824034+0000","last_undegraded":"2023-10-07T23:21:25.824034+0000","last_fullsized":"2023-10-07T23:21:25.824034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820528+0000","last_change":"2023-10-07T23:21:25.817564+0000","last_active":"2023-10-07T23:21:25.820528+0000","last_peered":"2023-10-07T23:21:25.820528+0000","last_clean":"2023-10-07T23:21:25.820528+0000","last_became_active":"2023-10-07T23:21:25.817149+0000","last_became_peered":"2023-10-07T23:21:25.817149+0000","last_unstale":"2023-10-07T23:21:25.820528+0000","last_undegraded":"2023-10-07T23:21:25.820528+0000","last_fullsized":"2023-10-07T23:21:25.820528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:31:32.392634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823467+0000","last_change":"2023-10-07T23:21:25.813864+0000","last_active":"2023-10-07T23:21:25.823467+0000","last_peered":"2023-10-07T23:21:25.823467+0000","last_clean":"2023-10-07T23:21:25.823467+0000","last_became_active":"2023-10-07T23:21:25.813627+0000","last_became_peered":"2023-10-07T23:21:25.813627+0000","last_unstale":"2023-10-07T23:21:25.823467+0000","last_undegraded":"2023-10-07T23:21:25.823467+0000","last_fullsized":"2023-10-07T23:21:25.823467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822927+0000","last_change":"2023-10-07T23:21:25.817746+0000","last_active":"2023-10-07T23:21:25.822927+0000","last_peered":"2023-10-07T23:21:25.822927+0000","last_clean":"2023-10-07T23:21:25.822927+0000","last_became_active":"2023-10-07T23:21:25.817419+0000","last_became_peered":"2023-10-07T23:21:25.817419+0000","last_unstale":"2023-10-07T23:21:25.822927+0000","last_undegraded":"2023-10-07T23:21:25.822927+0000","last_fullsized":"2023-10-07T23:21:25.822927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824136+0000","last_change":"2023-10-07T23:21:25.817774+0000","last_active":"2023-10-07T23:21:25.824136+0000","last_peered":"2023-10-07T23:21:25.824136+0000","last_clean":"2023-10-07T23:21:25.824136+0000","last_became_active":"2023-10-07T23:21:25.817279+0000","last_became_peered":"2023-10-07T23:21:25.817279+0000","last_unstale":"2023-10-07T23:21:25.824136+0000","last_undegraded":"2023-10-07T23:21:25.824136+0000","last_fullsized":"2023-10-07T23:21:25.824136+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824103+0000","last_change":"2023-10-07T23:21:25.821044+0000","last_active":"2023-10-07T23:21:25.824103+0000","last_peered":"2023-10-07T23:21:25.824103+0000","last_clean":"2023-10-07T23:21:25.824103+0000","last_became_active":"2023-10-07T23:21:25.820744+0000","last_became_peered":"2023-10-07T23:21:25.820744+0000","last_unstale":"2023-10-07T23:21:25.824103+0000","last_undegraded":"2023-10-07T23:21:25.824103+0000","last_fullsized":"2023-10-07T23:21:25.824103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.815381+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.815050+0000","last_became_peered":"2023-10-07T23:21:25.815050+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824873+0000","last_change":"2023-10-07T23:21:25.817306+0000","last_active":"2023-10-07T23:21:25.824873+0000","last_peered":"2023-10-07T23:21:25.824873+0000","last_clean":"2023-10-07T23:21:25.824873+0000","last_became_active":"2023-10-07T23:21:25.816989+0000","last_became_peered":"2023-10-07T23:21:25.816989+0000","last_unstale":"2023-10-07T23:21:25.824873+0000","last_undegraded":"2023-10-07T23:21:25.824873+0000","last_fullsized":"2023-10-07T23:21:25.824873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823714+0000","last_change":"2023-10-07T23:21:25.815302+0000","last_active":"2023-10-07T23:21:25.823714+0000","last_peered":"2023-10-07T23:21:25.823714+0000","last_clean":"2023-10-07T23:21:25.823714+0000","last_became_active":"2023-10-07T23:21:25.814105+0000","last_became_peered":"2023-10-07T23:21:25.814105+0000","last_unstale":"2023-10-07T23:21:25.823714+0000","last_undegraded":"2023-10-07T23:21:25.823714+0000","last_fullsized":"2023-10-07T23:21:25.823714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:51:37.211418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.805674+0000","last_change":"2023-10-07T23:21:25.800296+0000","last_active":"2023-10-07T23:21:25.805674+0000","last_peered":"2023-10-07T23:21:25.805674+0000","last_clean":"2023-10-07T23:21:25.805674+0000","last_became_active":"2023-10-07T23:21:25.799882+0000","last_became_peered":"2023-10-07T23:21:25.799882+0000","last_unstale":"2023-10-07T23:21:25.805674+0000","last_undegraded":"2023-10-07T23:21:25.805674+0000","last_fullsized":"2023-10-07T23:21:25.805674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806603+0000","last_change":"2023-10-07T23:21:25.803198+0000","last_active":"2023-10-07T23:21:25.806603+0000","last_peered":"2023-10-07T23:21:25.806603+0000","last_clean":"2023-10-07T23:21:25.806603+0000","last_became_active":"2023-10-07T23:21:25.802597+0000","last_became_peered":"2023-10-07T23:21:25.802597+0000","last_unstale":"2023-10-07T23:21:25.806603+0000","last_undegraded":"2023-10-07T23:21:25.806603+0000","last_fullsized":"2023-10-07T23:21:25.806603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816488+0000","last_change":"2023-10-07T23:21:25.809325+0000","last_active":"2023-10-07T23:21:25.816488+0000","last_peered":"2023-10-07T23:21:25.816488+0000","last_clean":"2023-10-07T23:21:25.816488+0000","last_became_active":"2023-10-07T23:21:25.808944+0000","last_became_peered":"2023-10-07T23:21:25.808944+0000","last_unstale":"2023-10-07T23:21:25.816488+0000","last_undegraded":"2023-10-07T23:21:25.816488+0000","last_fullsized":"2023-10-07T23:21:25.816488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:50.633181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822948+0000","last_change":"2023-10-07T23:21:25.815656+0000","last_active":"2023-10-07T23:21:25.822948+0000","last_peered":"2023-10-07T23:21:25.822948+0000","last_clean":"2023-10-07T23:21:25.822948+0000","last_became_active":"2023-10-07T23:21:25.815290+0000","last_became_peered":"2023-10-07T23:21:25.815290+0000","last_unstale":"2023-10-07T23:21:25.822948+0000","last_undegraded":"2023-10-07T23:21:25.822948+0000","last_fullsized":"2023-10-07T23:21:25.822948+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831315+0000","last_change":"2023-10-07T23:21:25.821314+0000","last_active":"2023-10-07T23:21:25.831315+0000","last_peered":"2023-10-07T23:21:25.831315+0000","last_clean":"2023-10-07T23:21:25.831315+0000","last_became_active":"2023-10-07T23:21:25.820716+0000","last_became_peered":"2023-10-07T23:21:25.820716+0000","last_unstale":"2023-10-07T23:21:25.831315+0000","last_undegraded":"2023-10-07T23:21:25.831315+0000","last_fullsized":"2023-10-07T23:21:25.831315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820420+0000","last_change":"2023-10-07T23:21:25.815145+0000","last_active":"2023-10-07T23:21:25.820420+0000","last_peered":"2023-10-07T23:21:25.820420+0000","last_clean":"2023-10-07T23:21:25.820420+0000","last_became_active":"2023-10-07T23:21:25.814843+0000","last_became_peered":"2023-10-07T23:21:25.814843+0000","last_unstale":"2023-10-07T23:21:25.820420+0000","last_undegraded":"2023-10-07T23:21:25.820420+0000","last_fullsized":"2023-10-07T23:21:25.820420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:37.712425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824534+0000","last_change":"2023-10-07T23:21:25.822953+0000","last_active":"2023-10-07T23:21:25.824534+0000","last_peered":"2023-10-07T23:21:25.824534+0000","last_clean":"2023-10-07T23:21:25.824534+0000","last_became_active":"2023-10-07T23:21:25.822567+0000","last_became_peered":"2023-10-07T23:21:25.822567+0000","last_unstale":"2023-10-07T23:21:25.824534+0000","last_undegraded":"2023-10-07T23:21:25.824534+0000","last_fullsized":"2023-10-07T23:21:25.824534+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.818161+0000","last_change":"2023-10-07T23:21:25.810275+0000","last_active":"2023-10-07T23:21:25.818161+0000","last_peered":"2023-10-07T23:21:25.818161+0000","last_clean":"2023-10-07T23:21:25.818161+0000","last_became_active":"2023-10-07T23:21:25.808608+0000","last_became_peered":"2023-10-07T23:21:25.808608+0000","last_unstale":"2023-10-07T23:21:25.818161+0000","last_undegraded":"2023-10-07T23:21:25.818161+0000","last_fullsized":"2023-10-07T23:21:25.818161+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.815069+0000","last_change":"2023-10-07T23:21:25.810226+0000","last_active":"2023-10-07T23:21:25.815069+0000","last_peered":"2023-10-07T23:21:25.815069+0000","last_clean":"2023-10-07T23:21:25.815069+0000","last_became_active":"2023-10-07T23:21:25.809894+0000","last_became_peered":"2023-10-07T23:21:25.809894+0000","last_unstale":"2023-10-07T23:21:25.815069+0000","last_undegraded":"2023-10-07T23:21:25.815069+0000","last_fullsized":"2023-10-07T23:21:25.815069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820902+0000","last_change":"2023-10-07T23:21:25.811504+0000","last_active":"2023-10-07T23:21:25.820902+0000","last_peered":"2023-10-07T23:21:25.820902+0000","last_clean":"2023-10-07T23:21:25.820902+0000","last_became_active":"2023-10-07T23:21:25.811171+0000","last_became_peered":"2023-10-07T23:21:25.811171+0000","last_unstale":"2023-10-07T23:21:25.820902+0000","last_undegraded":"2023-10-07T23:21:25.820902+0000","last_fullsized":"2023-10-07T23:21:25.820902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:07:44.716222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813279+0000","last_change":"2023-10-07T23:21:25.811409+0000","last_active":"2023-10-07T23:21:25.813279+0000","last_peered":"2023-10-07T23:21:25.813279+0000","last_clean":"2023-10-07T23:21:25.813279+0000","last_became_active":"2023-10-07T23:21:25.811110+0000","last_became_peered":"2023-10-07T23:21:25.811110+0000","last_unstale":"2023-10-07T23:21:25.813279+0000","last_undegraded":"2023-10-07T23:21:25.813279+0000","last_fullsized":"2023-10-07T23:21:25.813279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831600+0000","last_change":"2023-10-07T23:21:25.823226+0000","last_active":"2023-10-07T23:21:25.831600+0000","last_peered":"2023-10-07T23:21:25.831600+0000","last_clean":"2023-10-07T23:21:25.831600+0000","last_became_active":"2023-10-07T23:21:25.822718+0000","last_became_peered":"2023-10-07T23:21:25.822718+0000","last_unstale":"2023-10-07T23:21:25.831600+0000","last_undegraded":"2023-10-07T23:21:25.831600+0000","last_fullsized":"2023-10-07T23:21:25.831600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816264+0000","last_change":"2023-10-07T23:21:25.808833+0000","last_active":"2023-10-07T23:21:25.816264+0000","last_peered":"2023-10-07T23:21:25.816264+0000","last_clean":"2023-10-07T23:21:25.816264+0000","last_became_active":"2023-10-07T23:21:25.808473+0000","last_became_peered":"2023-10-07T23:21:25.808473+0000","last_unstale":"2023-10-07T23:21:25.816264+0000","last_undegraded":"2023-10-07T23:21:25.816264+0000","last_fullsized":"2023-10-07T23:21:25.816264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:02.893709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822506+0000","last_change":"2023-10-07T23:21:25.815523+0000","last_active":"2023-10-07T23:21:25.822506+0000","last_peered":"2023-10-07T23:21:25.822506+0000","last_clean":"2023-10-07T23:21:25.822506+0000","last_became_active":"2023-10-07T23:21:25.814775+0000","last_became_peered":"2023-10-07T23:21:25.814775+0000","last_unstale":"2023-10-07T23:21:25.822506+0000","last_undegraded":"2023-10-07T23:21:25.822506+0000","last_fullsized":"2023-10-07T23:21:25.822506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823146+0000","last_change":"2023-10-07T23:21:25.818446+0000","last_active":"2023-10-07T23:21:25.823146+0000","last_peered":"2023-10-07T23:21:25.823146+0000","last_clean":"2023-10-07T23:21:25.823146+0000","last_became_active":"2023-10-07T23:21:25.818155+0000","last_became_peered":"2023-10-07T23:21:25.818155+0000","last_unstale":"2023-10-07T23:21:25.823146+0000","last_undegraded":"2023-10-07T23:21:25.823146+0000","last_fullsized":"2023-10-07T23:21:25.823146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822397+0000","last_change":"2023-10-07T23:21:25.815227+0000","last_active":"2023-10-07T23:21:25.822397+0000","last_peered":"2023-10-07T23:21:25.822397+0000","last_clean":"2023-10-07T23:21:25.822397+0000","last_became_active":"2023-10-07T23:21:25.814723+0000","last_became_peered":"2023-10-07T23:21:25.814723+0000","last_unstale":"2023-10-07T23:21:25.822397+0000","last_undegraded":"2023-10-07T23:21:25.822397+0000","last_fullsized":"2023-10-07T23:21:25.822397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810308+0000","last_change":"2023-10-07T23:21:25.808952+0000","last_active":"2023-10-07T23:21:25.810308+0000","last_peered":"2023-10-07T23:21:25.810308+0000","last_clean":"2023-10-07T23:21:25.810308+0000","last_became_active":"2023-10-07T23:21:25.808651+0000","last_became_peered":"2023-10-07T23:21:25.808651+0000","last_unstale":"2023-10-07T23:21:25.810308+0000","last_undegraded":"2023-10-07T23:21:25.810308+0000","last_fullsized":"2023-10-07T23:21:25.810308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820498+0000","last_change":"2023-10-07T23:21:25.816989+0000","last_active":"2023-10-07T23:21:25.820498+0000","last_peered":"2023-10-07T23:21:25.820498+0000","last_clean":"2023-10-07T23:21:25.820498+0000","last_became_active":"2023-10-07T23:21:25.816637+0000","last_became_peered":"2023-10-07T23:21:25.816637+0000","last_unstale":"2023-10-07T23:21:25.820498+0000","last_undegraded":"2023-10-07T23:21:25.820498+0000","last_fullsized":"2023-10-07T23:21:25.820498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:34.060599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824888+0000","last_change":"2023-10-07T23:21:25.817393+0000","last_active":"2023-10-07T23:21:25.824888+0000","last_peered":"2023-10-07T23:21:25.824888+0000","last_clean":"2023-10-07T23:21:25.824888+0000","last_became_active":"2023-10-07T23:21:25.816812+0000","last_became_peered":"2023-10-07T23:21:25.816812+0000","last_unstale":"2023-10-07T23:21:25.824888+0000","last_undegraded":"2023-10-07T23:21:25.824888+0000","last_fullsized":"2023-10-07T23:21:25.824888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831230+0000","last_change":"2023-10-07T23:21:25.820991+0000","last_active":"2023-10-07T23:21:25.831230+0000","last_peered":"2023-10-07T23:21:25.831230+0000","last_clean":"2023-10-07T23:21:25.831230+0000","last_became_active":"2023-10-07T23:21:25.820689+0000","last_became_peered":"2023-10-07T23:21:25.820689+0000","last_unstale":"2023-10-07T23:21:25.831230+0000","last_undegraded":"2023-10-07T23:21:25.831230+0000","last_fullsized":"2023-10-07T23:21:25.831230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:24.802866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822891+0000","last_change":"2023-10-07T23:21:25.818034+0000","last_active":"2023-10-07T23:21:25.822891+0000","last_peered":"2023-10-07T23:21:25.822891+0000","last_clean":"2023-10-07T23:21:25.822891+0000","last_became_active":"2023-10-07T23:21:25.817671+0000","last_became_peered":"2023-10-07T23:21:25.817671+0000","last_unstale":"2023-10-07T23:21:25.822891+0000","last_undegraded":"2023-10-07T23:21:25.822891+0000","last_fullsized":"2023-10-07T23:21:25.822891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:56:34.189379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816324+0000","last_change":"2023-10-07T23:21:25.814531+0000","last_active":"2023-10-07T23:21:25.816324+0000","last_peered":"2023-10-07T23:21:25.816324+0000","last_clean":"2023-10-07T23:21:25.816324+0000","last_became_active":"2023-10-07T23:21:25.814176+0000","last_became_peered":"2023-10-07T23:21:25.814176+0000","last_unstale":"2023-10-07T23:21:25.816324+0000","last_undegraded":"2023-10-07T23:21:25.816324+0000","last_fullsized":"2023-10-07T23:21:25.816324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:03.964344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819009+0000","last_change":"2023-10-07T23:21:25.815377+0000","last_active":"2023-10-07T23:21:25.819009+0000","last_peered":"2023-10-07T23:21:25.819009+0000","last_clean":"2023-10-07T23:21:25.819009+0000","last_became_active":"2023-10-07T23:21:25.815081+0000","last_became_peered":"2023-10-07T23:21:25.815081+0000","last_unstale":"2023-10-07T23:21:25.819009+0000","last_undegraded":"2023-10-07T23:21:25.819009+0000","last_fullsized":"2023-10-07T23:21:25.819009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816446+0000","last_change":"2023-10-07T23:21:25.809589+0000","last_active":"2023-10-07T23:21:25.816446+0000","last_peered":"2023-10-07T23:21:25.816446+0000","last_clean":"2023-10-07T23:21:25.816446+0000","last_became_active":"2023-10-07T23:21:25.809194+0000","last_became_peered":"2023-10-07T23:21:25.809194+0000","last_unstale":"2023-10-07T23:21:25.816446+0000","last_undegraded":"2023-10-07T23:21:25.816446+0000","last_fullsized":"2023-10-07T23:21:25.816446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:35.323397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.959548+0000","last_change":"2023-10-07T23:21:24.961412+0000","last_active":"2023-10-07T23:21:25.959548+0000","last_peered":"2023-10-07T23:21:25.959548+0000","last_clean":"2023-10-07T23:21:25.959548+0000","last_became_active":"2023-10-07T23:21:24.961027+0000","last_became_peered":"2023-10-07T23:21:24.961027+0000","last_unstale":"2023-10-07T23:21:25.959548+0000","last_undegraded":"2023-10-07T23:21:25.959548+0000","last_fullsized":"2023-10-07T23:21:25.959548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:13.180905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.831277+0000","last_change":"2023-10-07T23:21:25.821055+0000","last_active":"2023-10-07T23:21:25.831277+0000","last_peered":"2023-10-07T23:21:25.831277+0000","last_clean":"2023-10-07T23:21:25.831277+0000","last_became_active":"2023-10-07T23:21:25.820736+0000","last_became_peered":"2023-10-07T23:21:25.820736+0000","last_unstale":"2023-10-07T23:21:25.831277+0000","last_undegraded":"2023-10-07T23:21:25.831277+0000","last_fullsized":"2023-10-07T23:21:25.831277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:06:58.740095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.952918+0000","last_change":"2023-10-07T23:21:24.946739+0000","last_active":"2023-10-07T23:21:25.952918+0000","last_peered":"2023-10-07T23:21:25.952918+0000","last_clean":"2023-10-07T23:21:25.952918+0000","last_became_active":"2023-10-07T23:21:24.946320+0000","last_became_peered":"2023-10-07T23:21:24.946320+0000","last_unstale":"2023-10-07T23:21:25.952918+0000","last_undegraded":"2023-10-07T23:21:25.952918+0000","last_fullsized":"2023-10-07T23:21:25.952918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.810406+0000","last_change":"2023-10-07T23:21:24.955775+0000","last_active":"2023-10-07T23:21:25.810406+0000","last_peered":"2023-10-07T23:21:25.810406+0000","last_clean":"2023-10-07T23:21:25.810406+0000","last_became_active":"2023-10-07T23:21:24.955487+0000","last_became_peered":"2023-10-07T23:21:24.955487+0000","last_unstale":"2023-10-07T23:21:25.810406+0000","last_undegraded":"2023-10-07T23:21:25.810406+0000","last_fullsized":"2023-10-07T23:21:25.810406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825497+0000","last_change":"2023-10-07T23:21:25.822494+0000","last_active":"2023-10-07T23:21:25.825497+0000","last_peered":"2023-10-07T23:21:25.825497+0000","last_clean":"2023-10-07T23:21:25.825497+0000","last_became_active":"2023-10-07T23:21:25.821860+0000","last_became_peered":"2023-10-07T23:21:25.821860+0000","last_unstale":"2023-10-07T23:21:25.825497+0000","last_undegraded":"2023-10-07T23:21:25.825497+0000","last_fullsized":"2023-10-07T23:21:25.825497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:30.379527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.796432+0000","last_change":"2023-10-07T23:21:24.949505+0000","last_active":"2023-10-07T23:21:25.796432+0000","last_peered":"2023-10-07T23:21:25.796432+0000","last_clean":"2023-10-07T23:21:25.796432+0000","last_became_active":"2023-10-07T23:21:24.949195+0000","last_became_peered":"2023-10-07T23:21:24.949195+0000","last_unstale":"2023-10-07T23:21:25.796432+0000","last_undegraded":"2023-10-07T23:21:25.796432+0000","last_fullsized":"2023-10-07T23:21:25.796432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820474+0000","last_change":"2023-10-07T23:21:25.816941+0000","last_active":"2023-10-07T23:21:25.820474+0000","last_peered":"2023-10-07T23:21:25.820474+0000","last_clean":"2023-10-07T23:21:25.820474+0000","last_became_active":"2023-10-07T23:21:25.816658+0000","last_became_peered":"2023-10-07T23:21:25.816658+0000","last_unstale":"2023-10-07T23:21:25.820474+0000","last_undegraded":"2023-10-07T23:21:25.820474+0000","last_fullsized":"2023-10-07T23:21:25.820474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:20.455188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794906+0000","last_change":"2023-10-07T23:21:24.953996+0000","last_active":"2023-10-07T23:21:25.794906+0000","last_peered":"2023-10-07T23:21:25.794906+0000","last_clean":"2023-10-07T23:21:25.794906+0000","last_became_active":"2023-10-07T23:21:24.953684+0000","last_became_peered":"2023-10-07T23:21:24.953684+0000","last_unstale":"2023-10-07T23:21:25.794906+0000","last_undegraded":"2023-10-07T23:21:25.794906+0000","last_fullsized":"2023-10-07T23:21:25.794906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816295+0000","last_change":"2023-10-07T23:21:25.809600+0000","last_active":"2023-10-07T23:21:25.816295+0000","last_peered":"2023-10-07T23:21:25.816295+0000","last_clean":"2023-10-07T23:21:25.816295+0000","last_became_active":"2023-10-07T23:21:25.808699+0000","last_became_peered":"2023-10-07T23:21:25.808699+0000","last_unstale":"2023-10-07T23:21:25.816295+0000","last_undegraded":"2023-10-07T23:21:25.816295+0000","last_fullsized":"2023-10-07T23:21:25.816295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.793286+0000","last_change":"2023-10-07T23:21:24.952022+0000","last_active":"2023-10-07T23:21:25.793286+0000","last_peered":"2023-10-07T23:21:25.793286+0000","last_clean":"2023-10-07T23:21:25.793286+0000","last_became_active":"2023-10-07T23:21:24.951291+0000","last_became_peered":"2023-10-07T23:21:24.951291+0000","last_unstale":"2023-10-07T23:21:25.793286+0000","last_undegraded":"2023-10-07T23:21:25.793286+0000","last_fullsized":"2023-10-07T23:21:25.793286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823818+0000","last_change":"2023-10-07T23:21:25.816148+0000","last_active":"2023-10-07T23:21:25.823818+0000","last_peered":"2023-10-07T23:21:25.823818+0000","last_clean":"2023-10-07T23:21:25.823818+0000","last_became_active":"2023-10-07T23:21:25.815705+0000","last_became_peered":"2023-10-07T23:21:25.815705+0000","last_unstale":"2023-10-07T23:21:25.823818+0000","last_undegraded":"2023-10-07T23:21:25.823818+0000","last_fullsized":"2023-10-07T23:21:25.823818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:23.605003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954881+0000","last_change":"2023-10-07T23:21:24.954956+0000","last_active":"2023-10-07T23:21:25.954881+0000","last_peered":"2023-10-07T23:21:25.954881+0000","last_clean":"2023-10-07T23:21:25.954881+0000","last_became_active":"2023-10-07T23:21:24.954596+0000","last_became_peered":"2023-10-07T23:21:24.954596+0000","last_unstale":"2023-10-07T23:21:25.954881+0000","last_undegraded":"2023-10-07T23:21:25.954881+0000","last_fullsized":"2023-10-07T23:21:25.954881+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:37:02.562726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816506+0000","last_change":"2023-10-07T23:21:25.809979+0000","last_active":"2023-10-07T23:21:25.816506+0000","last_peered":"2023-10-07T23:21:25.816506+0000","last_clean":"2023-10-07T23:21:25.816506+0000","last_became_active":"2023-10-07T23:21:25.809594+0000","last_became_peered":"2023-10-07T23:21:25.809594+0000","last_unstale":"2023-10-07T23:21:25.816506+0000","last_undegraded":"2023-10-07T23:21:25.816506+0000","last_fullsized":"2023-10-07T23:21:25.816506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:28.775950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.966427+0000","last_change":"2023-10-07T23:21:24.954704+0000","last_active":"2023-10-07T23:21:25.966427+0000","last_peered":"2023-10-07T23:21:25.966427+0000","last_clean":"2023-10-07T23:21:25.966427+0000","last_became_active":"2023-10-07T23:21:24.954416+0000","last_became_peered":"2023-10-07T23:21:24.954416+0000","last_unstale":"2023-10-07T23:21:25.966427+0000","last_undegraded":"2023-10-07T23:21:25.966427+0000","last_fullsized":"2023-10-07T23:21:25.966427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824095+0000","last_change":"2023-10-07T23:21:25.817538+0000","last_active":"2023-10-07T23:21:25.824095+0000","last_peered":"2023-10-07T23:21:25.824095+0000","last_clean":"2023-10-07T23:21:25.824095+0000","last_became_active":"2023-10-07T23:21:25.817209+0000","last_became_peered":"2023-10-07T23:21:25.817209+0000","last_unstale":"2023-10-07T23:21:25.824095+0000","last_undegraded":"2023-10-07T23:21:25.824095+0000","last_fullsized":"2023-10-07T23:21:25.824095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:11:33.580729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964849+0000","last_change":"2023-10-07T23:21:24.957350+0000","last_active":"2023-10-07T23:21:25.964849+0000","last_peered":"2023-10-07T23:21:25.964849+0000","last_clean":"2023-10-07T23:21:25.964849+0000","last_became_active":"2023-10-07T23:21:24.957009+0000","last_became_peered":"2023-10-07T23:21:24.957009+0000","last_unstale":"2023-10-07T23:21:25.964849+0000","last_undegraded":"2023-10-07T23:21:25.964849+0000","last_fullsized":"2023-10-07T23:21:25.964849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:58.493058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823798+0000","last_change":"2023-10-07T23:21:25.814807+0000","last_active":"2023-10-07T23:21:25.823798+0000","last_peered":"2023-10-07T23:21:25.823798+0000","last_clean":"2023-10-07T23:21:25.823798+0000","last_became_active":"2023-10-07T23:21:25.813589+0000","last_became_peered":"2023-10-07T23:21:25.813589+0000","last_unstale":"2023-10-07T23:21:25.823798+0000","last_undegraded":"2023-10-07T23:21:25.823798+0000","last_fullsized":"2023-10-07T23:21:25.823798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797030+0000","last_change":"2023-10-07T23:21:24.957704+0000","last_active":"2023-10-07T23:21:25.797030+0000","last_peered":"2023-10-07T23:21:25.797030+0000","last_clean":"2023-10-07T23:21:25.797030+0000","last_became_active":"2023-10-07T23:21:24.957447+0000","last_became_peered":"2023-10-07T23:21:24.957447+0000","last_unstale":"2023-10-07T23:21:25.797030+0000","last_undegraded":"2023-10-07T23:21:25.797030+0000","last_fullsized":"2023-10-07T23:21:25.797030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824121+0000","last_change":"2023-10-07T23:21:25.817566+0000","last_active":"2023-10-07T23:21:25.824121+0000","last_peered":"2023-10-07T23:21:25.824121+0000","last_clean":"2023-10-07T23:21:25.824121+0000","last_became_active":"2023-10-07T23:21:25.816898+0000","last_became_peered":"2023-10-07T23:21:25.816898+0000","last_unstale":"2023-10-07T23:21:25.824121+0000","last_undegraded":"2023-10-07T23:21:25.824121+0000","last_fullsized":"2023-10-07T23:21:25.824121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.794453+0000","last_change":"2023-10-07T23:21:24.948051+0000","last_active":"2023-10-07T23:21:25.794453+0000","last_peered":"2023-10-07T23:21:25.794453+0000","last_clean":"2023-10-07T23:21:25.794453+0000","last_became_active":"2023-10-07T23:21:24.947742+0000","last_became_peered":"2023-10-07T23:21:24.947742+0000","last_unstale":"2023-10-07T23:21:25.794453+0000","last_undegraded":"2023-10-07T23:21:25.794453+0000","last_fullsized":"2023-10-07T23:21:25.794453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819027+0000","last_change":"2023-10-07T23:21:25.815665+0000","last_active":"2023-10-07T23:21:25.819027+0000","last_peered":"2023-10-07T23:21:25.819027+0000","last_clean":"2023-10-07T23:21:25.819027+0000","last_became_active":"2023-10-07T23:21:25.815286+0000","last_became_peered":"2023-10-07T23:21:25.815286+0000","last_unstale":"2023-10-07T23:21:25.819027+0000","last_undegraded":"2023-10-07T23:21:25.819027+0000","last_fullsized":"2023-10-07T23:21:25.819027+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.791323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.960851+0000","last_change":"2023-10-07T23:21:24.952011+0000","last_active":"2023-10-07T23:21:25.960851+0000","last_peered":"2023-10-07T23:21:25.960851+0000","last_clean":"2023-10-07T23:21:25.960851+0000","last_became_active":"2023-10-07T23:21:24.951691+0000","last_became_peered":"2023-10-07T23:21:24.951691+0000","last_unstale":"2023-10-07T23:21:25.960851+0000","last_undegraded":"2023-10-07T23:21:25.960851+0000","last_fullsized":"2023-10-07T23:21:25.960851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823976+0000","last_change":"2023-10-07T23:21:25.817054+0000","last_active":"2023-10-07T23:21:25.823976+0000","last_peered":"2023-10-07T23:21:25.823976+0000","last_clean":"2023-10-07T23:21:25.823976+0000","last_became_active":"2023-10-07T23:21:25.816667+0000","last_became_peered":"2023-10-07T23:21:25.816667+0000","last_unstale":"2023-10-07T23:21:25.823976+0000","last_undegraded":"2023-10-07T23:21:25.823976+0000","last_fullsized":"2023-10-07T23:21:25.823976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.961478+0000","last_change":"2023-10-07T23:21:24.958951+0000","last_active":"2023-10-07T23:21:25.961478+0000","last_peered":"2023-10-07T23:21:25.961478+0000","last_clean":"2023-10-07T23:21:25.961478+0000","last_became_active":"2023-10-07T23:21:24.958632+0000","last_became_peered":"2023-10-07T23:21:24.958632+0000","last_unstale":"2023-10-07T23:21:25.961478+0000","last_undegraded":"2023-10-07T23:21:25.961478+0000","last_fullsized":"2023-10-07T23:21:25.961478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:29.730287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823870+0000","last_change":"2023-10-07T23:21:25.816914+0000","last_active":"2023-10-07T23:21:25.823870+0000","last_peered":"2023-10-07T23:21:25.823870+0000","last_clean":"2023-10-07T23:21:25.823870+0000","last_became_active":"2023-10-07T23:21:25.816515+0000","last_became_peered":"2023-10-07T23:21:25.816515+0000","last_unstale":"2023-10-07T23:21:25.823870+0000","last_undegraded":"2023-10-07T23:21:25.823870+0000","last_fullsized":"2023-10-07T23:21:25.823870+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.955373+0000","last_change":"2023-10-07T23:21:24.956912+0000","last_active":"2023-10-07T23:21:25.955373+0000","last_peered":"2023-10-07T23:21:25.955373+0000","last_clean":"2023-10-07T23:21:25.955373+0000","last_became_active":"2023-10-07T23:21:24.956588+0000","last_became_peered":"2023-10-07T23:21:24.956588+0000","last_unstale":"2023-10-07T23:21:25.955373+0000","last_undegraded":"2023-10-07T23:21:25.955373+0000","last_fullsized":"2023-10-07T23:21:25.955373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824203+0000","last_change":"2023-10-07T23:21:25.818653+0000","last_active":"2023-10-07T23:21:25.824203+0000","last_peered":"2023-10-07T23:21:25.824203+0000","last_clean":"2023-10-07T23:21:25.824203+0000","last_became_active":"2023-10-07T23:21:25.817833+0000","last_became_peered":"2023-10-07T23:21:25.817833+0000","last_unstale":"2023-10-07T23:21:25.824203+0000","last_undegraded":"2023-10-07T23:21:25.824203+0000","last_fullsized":"2023-10-07T23:21:25.824203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:26:53.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954656+0000","last_change":"2023-10-07T23:21:24.948082+0000","last_active":"2023-10-07T23:21:25.954656+0000","last_peered":"2023-10-07T23:21:25.954656+0000","last_clean":"2023-10-07T23:21:25.954656+0000","last_became_active":"2023-10-07T23:21:24.947759+0000","last_became_peered":"2023-10-07T23:21:24.947759+0000","last_unstale":"2023-10-07T23:21:25.954656+0000","last_undegraded":"2023-10-07T23:21:25.954656+0000","last_fullsized":"2023-10-07T23:21:25.954656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:11.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822945+0000","last_change":"2023-10-07T23:21:25.817833+0000","last_active":"2023-10-07T23:21:25.822945+0000","last_peered":"2023-10-07T23:21:25.822945+0000","last_clean":"2023-10-07T23:21:25.822945+0000","last_became_active":"2023-10-07T23:21:25.817547+0000","last_became_peered":"2023-10-07T23:21:25.817547+0000","last_unstale":"2023-10-07T23:21:25.822945+0000","last_undegraded":"2023-10-07T23:21:25.822945+0000","last_fullsized":"2023-10-07T23:21:25.822945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:29:44.907483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.807532+0000","last_change":"2023-10-07T23:21:24.956344+0000","last_active":"2023-10-07T23:21:25.807532+0000","last_peered":"2023-10-07T23:21:25.807532+0000","last_clean":"2023-10-07T23:21:25.807532+0000","last_became_active":"2023-10-07T23:21:24.955974+0000","last_became_peered":"2023-10-07T23:21:24.955974+0000","last_unstale":"2023-10-07T23:21:25.807532+0000","last_undegraded":"2023-10-07T23:21:25.807532+0000","last_fullsized":"2023-10-07T23:21:25.807532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823762+0000","last_change":"2023-10-07T23:21:25.816194+0000","last_active":"2023-10-07T23:21:25.823762+0000","last_peered":"2023-10-07T23:21:25.823762+0000","last_clean":"2023-10-07T23:21:25.823762+0000","last_became_active":"2023-10-07T23:21:25.813965+0000","last_became_peered":"2023-10-07T23:21:25.813965+0000","last_unstale":"2023-10-07T23:21:25.823762+0000","last_undegraded":"2023-10-07T23:21:25.823762+0000","last_fullsized":"2023-10-07T23:21:25.823762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:58:16.630926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802932+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:21:25.802932+0000","last_peered":"2023-10-07T23:21:25.802932+0000","last_clean":"2023-10-07T23:21:25.802932+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:21:25.802932+0000","last_undegraded":"2023-10-07T23:21:25.802932+0000","last_fullsized":"2023-10-07T23:21:25.802932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.792857+0000","last_change":"2023-10-07T23:21:24.953534+0000","last_active":"2023-10-07T23:21:25.792857+0000","last_peered":"2023-10-07T23:21:25.792857+0000","last_clean":"2023-10-07T23:21:25.792857+0000","last_became_active":"2023-10-07T23:21:24.953202+0000","last_became_peered":"2023-10-07T23:21:24.953202+0000","last_unstale":"2023-10-07T23:21:25.792857+0000","last_undegraded":"2023-10-07T23:21:25.792857+0000","last_fullsized":"2023-10-07T23:21:25.792857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824763+0000","last_change":"2023-10-07T23:21:25.816874+0000","last_active":"2023-10-07T23:21:25.824763+0000","last_peered":"2023-10-07T23:21:25.824763+0000","last_clean":"2023-10-07T23:21:25.824763+0000","last_became_active":"2023-10-07T23:21:25.816098+0000","last_became_peered":"2023-10-07T23:21:25.816098+0000","last_unstale":"2023-10-07T23:21:25.824763+0000","last_undegraded":"2023-10-07T23:21:25.824763+0000","last_fullsized":"2023-10-07T23:21:25.824763+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:23:07.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802840+0000","last_change":"2023-10-07T23:21:24.962711+0000","last_active":"2023-10-07T23:21:25.802840+0000","last_peered":"2023-10-07T23:21:25.802840+0000","last_clean":"2023-10-07T23:21:25.802840+0000","last_became_active":"2023-10-07T23:21:24.962404+0000","last_became_peered":"2023-10-07T23:21:24.962404+0000","last_unstale":"2023-10-07T23:21:25.802840+0000","last_undegraded":"2023-10-07T23:21:25.802840+0000","last_fullsized":"2023-10-07T23:21:25.802840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:59:02.950946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.819414+0000","last_change":"2023-10-07T23:21:25.809376+0000","last_active":"2023-10-07T23:21:25.819414+0000","last_peered":"2023-10-07T23:21:25.819414+0000","last_clean":"2023-10-07T23:21:25.819414+0000","last_became_active":"2023-10-07T23:21:25.809055+0000","last_became_peered":"2023-10-07T23:21:25.809055+0000","last_unstale":"2023-10-07T23:21:25.819414+0000","last_undegraded":"2023-10-07T23:21:25.819414+0000","last_fullsized":"2023-10-07T23:21:25.819414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.806772+0000","last_change":"2023-10-07T23:21:24.960450+0000","last_active":"2023-10-07T23:21:25.806772+0000","last_peered":"2023-10-07T23:21:25.806772+0000","last_clean":"2023-10-07T23:21:25.806772+0000","last_became_active":"2023-10-07T23:21:24.960068+0000","last_became_peered":"2023-10-07T23:21:24.960068+0000","last_unstale":"2023-10-07T23:21:25.806772+0000","last_undegraded":"2023-10-07T23:21:25.806772+0000","last_fullsized":"2023-10-07T23:21:25.806772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:36:31.559622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820366+0000","last_change":"2023-10-07T23:21:25.815004+0000","last_active":"2023-10-07T23:21:25.820366+0000","last_peered":"2023-10-07T23:21:25.820366+0000","last_clean":"2023-10-07T23:21:25.820366+0000","last_became_active":"2023-10-07T23:21:25.814688+0000","last_became_peered":"2023-10-07T23:21:25.814688+0000","last_unstale":"2023-10-07T23:21:25.820366+0000","last_undegraded":"2023-10-07T23:21:25.820366+0000","last_fullsized":"2023-10-07T23:21:25.820366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:46.190235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.797147+0000","last_change":"2023-10-07T23:21:24.952189+0000","last_active":"2023-10-07T23:21:25.797147+0000","last_peered":"2023-10-07T23:21:25.797147+0000","last_clean":"2023-10-07T23:21:25.797147+0000","last_became_active":"2023-10-07T23:21:24.951731+0000","last_became_peered":"2023-10-07T23:21:24.951731+0000","last_unstale":"2023-10-07T23:21:25.797147+0000","last_undegraded":"2023-10-07T23:21:25.797147+0000","last_fullsized":"2023-10-07T23:21:25.797147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823922+0000","last_change":"2023-10-07T23:21:25.822875+0000","last_active":"2023-10-07T23:21:25.823922+0000","last_peered":"2023-10-07T23:21:25.823922+0000","last_clean":"2023-10-07T23:21:25.823922+0000","last_became_active":"2023-10-07T23:21:25.822536+0000","last_became_peered":"2023-10-07T23:21:25.822536+0000","last_unstale":"2023-10-07T23:21:25.823922+0000","last_undegraded":"2023-10-07T23:21:25.823922+0000","last_fullsized":"2023-10-07T23:21:25.823922+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:47.179786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.800662+0000","last_change":"2023-10-07T23:21:24.952562+0000","last_active":"2023-10-07T23:21:25.800662+0000","last_peered":"2023-10-07T23:21:25.800662+0000","last_clean":"2023-10-07T23:21:25.800662+0000","last_became_active":"2023-10-07T23:21:24.952071+0000","last_became_peered":"2023-10-07T23:21:24.952071+0000","last_unstale":"2023-10-07T23:21:25.800662+0000","last_undegraded":"2023-10-07T23:21:25.800662+0000","last_fullsized":"2023-10-07T23:21:25.800662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:17.603304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.822867+0000","last_change":"2023-10-07T23:21:25.818214+0000","last_active":"2023-10-07T23:21:25.822867+0000","last_peered":"2023-10-07T23:21:25.822867+0000","last_clean":"2023-10-07T23:21:25.822867+0000","last_became_active":"2023-10-07T23:21:25.817888+0000","last_became_peered":"2023-10-07T23:21:25.817888+0000","last_unstale":"2023-10-07T23:21:25.822867+0000","last_undegraded":"2023-10-07T23:21:25.822867+0000","last_fullsized":"2023-10-07T23:21:25.822867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:58.986701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.804473+0000","last_change":"2023-10-07T23:21:24.962199+0000","last_active":"2023-10-07T23:21:25.804473+0000","last_peered":"2023-10-07T23:21:25.804473+0000","last_clean":"2023-10-07T23:21:25.804473+0000","last_became_active":"2023-10-07T23:21:24.961778+0000","last_became_peered":"2023-10-07T23:21:24.961778+0000","last_unstale":"2023-10-07T23:21:25.804473+0000","last_undegraded":"2023-10-07T23:21:25.804473+0000","last_fullsized":"2023-10-07T23:21:25.804473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.820428+0000","last_change":"2023-10-07T23:21:25.815292+0000","last_active":"2023-10-07T23:21:25.820428+0000","last_peered":"2023-10-07T23:21:25.820428+0000","last_clean":"2023-10-07T23:21:25.820428+0000","last_became_active":"2023-10-07T23:21:25.814294+0000","last_became_peered":"2023-10-07T23:21:25.814294+0000","last_unstale":"2023-10-07T23:21:25.820428+0000","last_undegraded":"2023-10-07T23:21:25.820428+0000","last_fullsized":"2023-10-07T23:21:25.820428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:41.638201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-07T23:21:27.097216+0000","last_change":"2023-10-07T23:21:24.948131+0000","last_active":"2023-10-07T23:21:27.097216+0000","last_peered":"2023-10-07T23:21:27.097216+0000","last_clean":"2023-10-07T23:21:27.097216+0000","last_became_active":"2023-10-07T23:21:24.947831+0000","last_became_peered":"2023-10-07T23:21:24.947831+0000","last_unstale":"2023-10-07T23:21:27.097216+0000","last_undegraded":"2023-10-07T23:21:27.097216+0000","last_fullsized":"2023-10-07T23:21:27.097216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:23.995859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825605+0000","last_change":"2023-10-07T23:21:25.823117+0000","last_active":"2023-10-07T23:21:25.825605+0000","last_peered":"2023-10-07T23:21:25.825605+0000","last_clean":"2023-10-07T23:21:25.825605+0000","last_became_active":"2023-10-07T23:21:25.822814+0000","last_became_peered":"2023-10-07T23:21:25.822814+0000","last_unstale":"2023-10-07T23:21:25.825605+0000","last_undegraded":"2023-10-07T23:21:25.825605+0000","last_fullsized":"2023-10-07T23:21:25.825605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954544+0000","last_change":"2023-10-07T23:21:24.957129+0000","last_active":"2023-10-07T23:21:25.954544+0000","last_peered":"2023-10-07T23:21:25.954544+0000","last_clean":"2023-10-07T23:21:25.954544+0000","last_became_active":"2023-10-07T23:21:24.956815+0000","last_became_peered":"2023-10-07T23:21:24.956815+0000","last_unstale":"2023-10-07T23:21:25.954544+0000","last_undegraded":"2023-10-07T23:21:25.954544+0000","last_fullsized":"2023-10-07T23:21:25.954544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:01:49.619784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823723+0000","last_change":"2023-10-07T23:21:25.814482+0000","last_active":"2023-10-07T23:21:25.823723+0000","last_peered":"2023-10-07T23:21:25.823723+0000","last_clean":"2023-10-07T23:21:25.823723+0000","last_became_active":"2023-10-07T23:21:25.814126+0000","last_became_peered":"2023-10-07T23:21:25.814126+0000","last_unstale":"2023-10-07T23:21:25.823723+0000","last_undegraded":"2023-10-07T23:21:25.823723+0000","last_fullsized":"2023-10-07T23:21:25.823723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.802684+0000","last_change":"2023-10-07T23:21:24.960263+0000","last_active":"2023-10-07T23:21:25.802684+0000","last_peered":"2023-10-07T23:21:25.802684+0000","last_clean":"2023-10-07T23:21:25.802684+0000","last_became_active":"2023-10-07T23:21:24.959984+0000","last_became_peered":"2023-10-07T23:21:24.959984+0000","last_unstale":"2023-10-07T23:21:25.802684+0000","last_undegraded":"2023-10-07T23:21:25.802684+0000","last_fullsized":"2023-10-07T23:21:25.802684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:34.309999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.821174+0000","last_change":"2023-10-07T23:21:25.819404+0000","last_active":"2023-10-07T23:21:25.821174+0000","last_peered":"2023-10-07T23:21:25.821174+0000","last_clean":"2023-10-07T23:21:25.821174+0000","last_became_active":"2023-10-07T23:21:25.818658+0000","last_became_peered":"2023-10-07T23:21:25.818658+0000","last_unstale":"2023-10-07T23:21:25.821174+0000","last_undegraded":"2023-10-07T23:21:25.821174+0000","last_fullsized":"2023-10-07T23:21:25.821174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:28.390107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.799830+0000","last_change":"2023-10-07T23:21:24.958141+0000","last_active":"2023-10-07T23:21:25.799830+0000","last_peered":"2023-10-07T23:21:25.799830+0000","last_clean":"2023-10-07T23:21:25.799830+0000","last_became_active":"2023-10-07T23:21:24.957791+0000","last_became_peered":"2023-10-07T23:21:24.957791+0000","last_unstale":"2023-10-07T23:21:25.799830+0000","last_undegraded":"2023-10-07T23:21:25.799830+0000","last_fullsized":"2023-10-07T23:21:25.799830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:01.467025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825628+0000","last_change":"2023-10-07T23:21:25.823525+0000","last_active":"2023-10-07T23:21:25.825628+0000","last_peered":"2023-10-07T23:21:25.825628+0000","last_clean":"2023-10-07T23:21:25.825628+0000","last_became_active":"2023-10-07T23:21:25.822193+0000","last_became_peered":"2023-10-07T23:21:25.822193+0000","last_unstale":"2023-10-07T23:21:25.825628+0000","last_undegraded":"2023-10-07T23:21:25.825628+0000","last_fullsized":"2023-10-07T23:21:25.825628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.798501+0000","last_change":"2023-10-07T23:21:24.957341+0000","last_active":"2023-10-07T23:21:25.798501+0000","last_peered":"2023-10-07T23:21:25.798501+0000","last_clean":"2023-10-07T23:21:25.798501+0000","last_became_active":"2023-10-07T23:21:24.957059+0000","last_became_peered":"2023-10-07T23:21:24.957059+0000","last_unstale":"2023-10-07T23:21:25.798501+0000","last_undegraded":"2023-10-07T23:21:25.798501+0000","last_fullsized":"2023-10-07T23:21:25.798501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:34.702939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.825528+0000","last_change":"2023-10-07T23:21:25.822814+0000","last_active":"2023-10-07T23:21:25.825528+0000","last_peered":"2023-10-07T23:21:25.825528+0000","last_clean":"2023-10-07T23:21:25.825528+0000","last_became_active":"2023-10-07T23:21:25.822297+0000","last_became_peered":"2023-10-07T23:21:25.822297+0000","last_unstale":"2023-10-07T23:21:25.825528+0000","last_undegraded":"2023-10-07T23:21:25.825528+0000","last_fullsized":"2023-10-07T23:21:25.825528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:48.964596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954772+0000","last_change":"2023-10-07T23:21:24.961807+0000","last_active":"2023-10-07T23:21:25.954772+0000","last_peered":"2023-10-07T23:21:25.954772+0000","last_clean":"2023-10-07T23:21:25.954772+0000","last_became_active":"2023-10-07T23:21:24.961451+0000","last_became_peered":"2023-10-07T23:21:24.961451+0000","last_unstale":"2023-10-07T23:21:25.954772+0000","last_undegraded":"2023-10-07T23:21:25.954772+0000","last_fullsized":"2023-10-07T23:21:25.954772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:46.166388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.813287+0000","last_change":"2023-10-07T23:21:25.811192+0000","last_active":"2023-10-07T23:21:25.813287+0000","last_peered":"2023-10-07T23:21:25.813287+0000","last_clean":"2023-10-07T23:21:25.813287+0000","last_became_active":"2023-10-07T23:21:25.810877+0000","last_became_peered":"2023-10-07T23:21:25.810877+0000","last_unstale":"2023-10-07T23:21:25.813287+0000","last_undegraded":"2023-10-07T23:21:25.813287+0000","last_fullsized":"2023-10-07T23:21:25.813287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.816476+0000","last_change":"2023-10-07T23:21:25.811521+0000","last_active":"2023-10-07T23:21:25.816476+0000","last_peered":"2023-10-07T23:21:25.816476+0000","last_clean":"2023-10-07T23:21:25.816476+0000","last_became_active":"2023-10-07T23:21:25.811256+0000","last_became_peered":"2023-10-07T23:21:25.811256+0000","last_unstale":"2023-10-07T23:21:25.816476+0000","last_undegraded":"2023-10-07T23:21:25.816476+0000","last_fullsized":"2023-10-07T23:21:25.816476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:09.082812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954460+0000","last_change":"2023-10-07T23:21:24.962057+0000","last_active":"2023-10-07T23:21:25.954460+0000","last_peered":"2023-10-07T23:21:25.954460+0000","last_clean":"2023-10-07T23:21:25.954460+0000","last_became_active":"2023-10-07T23:21:24.961540+0000","last_became_peered":"2023-10-07T23:21:24.961540+0000","last_unstale":"2023-10-07T23:21:25.954460+0000","last_undegraded":"2023-10-07T23:21:25.954460+0000","last_fullsized":"2023-10-07T23:21:25.954460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:13:27.762552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.817631+0000","last_change":"2023-10-07T23:21:25.812184+0000","last_active":"2023-10-07T23:21:25.817631+0000","last_peered":"2023-10-07T23:21:25.817631+0000","last_clean":"2023-10-07T23:21:25.817631+0000","last_became_active":"2023-10-07T23:21:25.811844+0000","last_became_peered":"2023-10-07T23:21:25.811844+0000","last_unstale":"2023-10-07T23:21:25.817631+0000","last_undegraded":"2023-10-07T23:21:25.817631+0000","last_fullsized":"2023-10-07T23:21:25.817631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:55.374716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.954389+0000","last_change":"2023-10-07T23:21:24.961653+0000","last_active":"2023-10-07T23:21:25.954389+0000","last_peered":"2023-10-07T23:21:25.954389+0000","last_clean":"2023-10-07T23:21:25.954389+0000","last_became_active":"2023-10-07T23:21:24.961362+0000","last_became_peered":"2023-10-07T23:21:24.961362+0000","last_unstale":"2023-10-07T23:21:25.954389+0000","last_undegraded":"2023-10-07T23:21:25.954389+0000","last_fullsized":"2023-10-07T23:21:25.954389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:27.376709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.823793+0000","last_change":"2023-10-07T23:21:25.813813+0000","last_active":"2023-10-07T23:21:25.823793+0000","last_peered":"2023-10-07T23:21:25.823793+0000","last_clean":"2023-10-07T23:21:25.823793+0000","last_became_active":"2023-10-07T23:21:25.813500+0000","last_became_peered":"2023-10-07T23:21:25.813500+0000","last_unstale":"2023-10-07T23:21:25.823793+0000","last_undegraded":"2023-10-07T23:21:25.823793+0000","last_fullsized":"2023-10-07T23:21:25.823793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:05.735364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.965038+0000","last_change":"2023-10-07T23:21:24.959465+0000","last_active":"2023-10-07T23:21:25.965038+0000","last_peered":"2023-10-07T23:21:25.965038+0000","last_clean":"2023-10-07T23:21:25.965038+0000","last_became_active":"2023-10-07T23:21:24.957700+0000","last_became_peered":"2023-10-07T23:21:24.957700+0000","last_unstale":"2023-10-07T23:21:25.965038+0000","last_undegraded":"2023-10-07T23:21:25.965038+0000","last_fullsized":"2023-10-07T23:21:25.965038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:52:54.324670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.824672+0000","last_change":"2023-10-07T23:21:25.816098+0000","last_active":"2023-10-07T23:21:25.824672+0000","last_peered":"2023-10-07T23:21:25.824672+0000","last_clean":"2023-10-07T23:21:25.824672+0000","last_became_active":"2023-10-07T23:21:25.815533+0000","last_became_peered":"2023-10-07T23:21:25.815533+0000","last_unstale":"2023-10-07T23:21:25.824672+0000","last_undegraded":"2023-10-07T23:21:25.824672+0000","last_fullsized":"2023-10-07T23:21:25.824672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:24.912407+0000","last_clean_scrub_stamp":"2023-10-07T23:21:24.912407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:40:07.777770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-07T23:21:25.964792+0000","last_change":"2023-10-07T23:21:24.960874+0000","last_active":"2023-10-07T23:21:25.964792+0000","last_peered":"2023-10-07T23:21:25.964792+0000","last_clean":"2023-10-07T23:21:25.964792+0000","last_became_active":"2023-10-07T23:21:24.960568+0000","last_became_peered":"2023-10-07T23:21:24.960568+0000","last_unstale":"2023-10-07T23:21:25.964792+0000","last_undegraded":"2023-10-07T23:21:25.964792+0000","last_fullsized":"2023-10-07T23:21:25.964792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:23.899009+0000","last_clean_scrub_stamp":"2023-10-07T23:21:23.899009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:14:44.479522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298100,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073740,"statfs":{"total":96636764160,"available":96331509760,"internally_reserved":0,"allocated":831488,"data_stored":671553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":82167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":80757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301624,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070216,"statfs":{"total":96636764160,"available":96327901184,"internally_reserved":0,"allocated":245760,"data_stored":82167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301612,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070228,"statfs":{"total":96636764160,"available":96327913472,"internally_reserved":0,"allocated":233472,"data_stored":80631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302204,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069636,"statfs":{"total":96636764160,"available":96327307264,"internally_reserved":0,"allocated":839680,"data_stored":671609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301620,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070220,"statfs":{"total":96636764160,"available":96327905280,"internally_reserved":0,"allocated":241664,"data_stored":80791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:21:33.276 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-07T23:21:33.276 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-07T23:21:33.276 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-07T23:21:33.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-07T23:21:33.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 Processor -- start 2023-10-07T23:21:33.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 -- start start 2023-10-07T23:21:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1efc0cbe50 0x7f1efc06af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc06a540 con 0x7f1efc0abf90 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc06ab00 con 0x7f1efc0cbe50 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f02fa4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0cc390 con 0x7f1efc0ac360 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f027a3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1efc0cbe50 0x7f1efc06af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57146/0 (socket says 172.21.15.57:57146) 2023-10-07T23:21:33.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 -- 172.21.15.57:0/1432992066 learned_addr learned my addr 172.21.15.57:0/1432992066 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:33.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 -- 172.21.15.57:0/1432992066 >> v2:172.21.15.154:3300/0 conn(0x7f1efc0cbe50 msgr2=0x7f1efc06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 --2- 172.21.15.57:0/1432992066 >> v2:172.21.15.154:3300/0 conn(0x7f1efc0cbe50 0x7f1efc06af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 -- 172.21.15.57:0/1432992066 >> v1:172.21.15.154:6789/0 conn(0x7f1efc0abf90 legacy=0x7f1efc0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 -- 172.21.15.57:0/1432992066 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0cc670 con 0x7f1efc0ac360 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.370+0000 7f1f017a1700 1 --2- 172.21.15.57:0/1432992066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00099b0 tx=0x7f1ef0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b5b2ea4f13c452f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f00fa0700 1 -- 172.21.15.57:0/1432992066 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0009ea0 con 0x7f1efc0ac360 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f00fa0700 1 -- 172.21.15.57:0/1432992066 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef000b860 con 0x7f1efc0ac360 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f00fa0700 1 -- 172.21.15.57:0/1432992066 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000bc00 con 0x7f1efc0ac360 2023-10-07T23:21:33.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- 172.21.15.57:0/1432992066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 msgr2=0x7f1efc0cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/1432992066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00099b0 tx=0x7f1ef0009250 comp rx=0 tx=0).stop 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- 172.21.15.57:0/1432992066 shutdown_connections 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/1432992066 >> v2:172.21.15.154:3300/0 conn(0x7f1efc0cbe50 0x7f1efc06af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/1432992066 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc0cb910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- 172.21.15.57:0/1432992066 >> 172.21.15.57:0/1432992066 conn(0x7f1efc0670e0 msgr2=0x7f1efc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- 172.21.15.57:0/1432992066 shutdown_connections 2023-10-07T23:21:33.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- 172.21.15.57:0/1432992066 wait complete. 2023-10-07T23:21:33.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 Processor -- start 2023-10-07T23:21:33.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- start start 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 0x7f1efc148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c5fc0 con 0x7f1efc0cbe50 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c6260 con 0x7f1efc0abf90 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f02fa4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc151be0 con 0x7f1efc0ac360 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f027a3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 0x7f1efc148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f027a3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58656/0 (socket says 172.21.15.57:58656) 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f027a3700 1 -- 172.21.15.57:0/952664686 learned_addr learned my addr 172.21.15.57:0/952664686 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f017a1700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 -- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 msgr2=0x7f1efc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 -- 172.21.15.57:0/952664686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 msgr2=0x7f1efc148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f01fa2700 1 -- 172.21.15.57:0/952664686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00094f0 con 0x7f1efc0abf90 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.374+0000 7f1f027a3700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1f01fa2700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 0x7f1efc148e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c590 tx=0x7f1ef8007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011590 con 0x7f1efc0abf90 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8011710 con 0x7f1efc0abf90 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc151ec0 con 0x7f1efc0abf90 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011ad0 con 0x7f1efc0abf90 2023-10-07T23:21:33.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc151820 con 0x7f1efc0abf90 2023-10-07T23:21:33.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc1523c0 con 0x7f1efc0abf90 2023-10-07T23:21:33.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.378+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1ef801b3b0 con 0x7f1efc0abf90 2023-10-07T23:21:33.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.382+0000 7f1eedffb700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 0x7f1ee0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.382+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ef807bf90 con 0x7f1efc0abf90 2023-10-07T23:21:33.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.382+0000 7f1f017a1700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 0x7f1ee0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.382+0000 7f1f017a1700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 0x7f1ee0062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00099b0 tx=0x7f1ef0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.386+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ef804c600 con 0x7f1efc0abf90 2023-10-07T23:21:33.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.574+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1efc04be40 con 0x7f1efc0abf90 2023-10-07T23:21:33.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.574+0000 7f1eedffb700 1 -- 172.21.15.57:0/952664686 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1ef8017020 con 0x7f1efc0abf90 2023-10-07T23:21:33.581 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:33.581 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-07T23:21:33.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 msgr2=0x7f1ee0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 0x7f1ee0062a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00099b0 tx=0x7f1ef0017000 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 msgr2=0x7f1efc148e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 0x7f1efc148e70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c590 tx=0x7f1ef8007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 shutdown_connections 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1efc0cbe50 0x7f1efc148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1efc0ac360 0x7f1efc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ee0060550 0x7f1ee0062a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 --2- 172.21.15.57:0/952664686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1efc0abf90 0x7f1efc148e70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 >> 172.21.15.57:0/952664686 conn(0x7f1efc0670e0 msgr2=0x7f1efc154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 shutdown_connections 2023-10-07T23:21:33.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.578+0000 7f1f02fa4700 1 -- 172.21.15.57:0/952664686 wait complete. 2023-10-07T23:21:33.599 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-07T23:21:33.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:33.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 Processor -- start 2023-10-07T23:21:33.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 -- start start 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcde8159130 0x7fcde80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80c83b0 con 0x7fcde806bc90 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde80c8690 con 0x7fcde8158d60 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcdedf07700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80c81d0 con 0x7fcde8159130 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcded706700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcde8159130 0x7fcde80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcde7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcde806bc90 0x7fcde80c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60570/0 (socket says 172.21.15.57:60570) 2023-10-07T23:21:33.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcde7fff700 1 -- 172.21.15.57:0/2503382774 learned_addr learned my addr 172.21.15.57:0/2503382774 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcded706700 1 -- 172.21.15.57:0/2503382774 >> v1:172.21.15.154:6789/0 conn(0x7fcde8158d60 legacy=0x7fcde806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcded706700 1 -- 172.21.15.57:0/2503382774 >> v1:172.21.15.57:6789/0 conn(0x7fcde806bc90 legacy=0x7fcde80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcde77fe700 1 -- 172.21.15.57:0/2503382774 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132011716 0 0) 0x7fcde80c83b0 con 0x7fcde806bc90 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcded706700 1 -- 172.21.15.57:0/2503382774 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80c8970 con 0x7fcde8159130 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.694+0000 7fcded706700 1 --2- 172.21.15.57:0/2503382774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8159130 0x7fcde80c7c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009e60 tx=0x7fcde0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75514cc003c83015 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcde77fe700 1 -- 172.21.15.57:0/2503382774 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000e070 con 0x7fcde8159130 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcde77fe700 1 -- 172.21.15.57:0/2503382774 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000bc30 con 0x7fcde8159130 2023-10-07T23:21:33.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcde77fe700 1 -- 172.21.15.57:0/2503382774 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0012430 con 0x7fcde8159130 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- 172.21.15.57:0/2503382774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8159130 msgr2=0x7fcde80c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2503382774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8159130 0x7fcde80c7c90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009e60 tx=0x7fcde0009780 comp rx=0 tx=0).stop 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- 172.21.15.57:0/2503382774 shutdown_connections 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2503382774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8159130 0x7fcde80c7c90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- 172.21.15.57:0/2503382774 >> 172.21.15.57:0/2503382774 conn(0x7fcde80670e0 msgr2=0x7fcde80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- 172.21.15.57:0/2503382774 shutdown_connections 2023-10-07T23:21:33.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- 172.21.15.57:0/2503382774 wait complete. 2023-10-07T23:21:33.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 Processor -- start 2023-10-07T23:21:33.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.698+0000 7fcdedf07700 1 -- start start 2023-10-07T23:21:33.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcde806bc90 0x7fcde80adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 0x7fcde80c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b0cb0 con 0x7fcde8158d60 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b00a0 con 0x7fcde8159130 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b0380 con 0x7fcde806bc90 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58668/0 (socket says 172.21.15.57:58668) 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 -- 172.21.15.57:0/2130591792 learned_addr learned my addr 172.21.15.57:0/2130591792 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdecf05700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcde806bc90 0x7fcde80adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcded706700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 0x7fcde80c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 -- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcde806bc90 msgr2=0x7fcde80adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcde806bc90 0x7fcde80adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 -- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 msgr2=0x7fcde80c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 0x7fcde80c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 -- 172.21.15.57:0/2130591792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009a20 con 0x7fcde8158d60 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcded706700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 0x7fcde80c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde7fff700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c5b0 tx=0x7fcdd4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4005b30 con 0x7fcde8158d60 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4005c90 con 0x7fcde8158d60 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4007320 con 0x7fcde8158d60 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80b0660 con 0x7fcde8158d60 2023-10-07T23:21:33.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.702+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80afd30 con 0x7fcde8158d60 2023-10-07T23:21:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde80711c0 con 0x7fcde8158d60 2023-10-07T23:21:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcdd401b3b0 con 0x7fcde8158d60 2023-10-07T23:21:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcde4ff9700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 0x7fcdcc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcdd407bc80 con 0x7fcde8158d60 2023-10-07T23:21:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcdecf05700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 0x7fcdcc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.706+0000 7fcdecf05700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 0x7fcdcc062a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003aa0 tx=0x7fcddc000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.714+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcdd404c270 con 0x7fcde8158d60 2023-10-07T23:21:33.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.870+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde80c8c50 con 0x7fcde8158d60 2023-10-07T23:21:33.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.870+0000 7fcde4ff9700 1 -- 172.21.15.57:0/2130591792 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fcdd404f890 con 0x7fcde8158d60 2023-10-07T23:21:33.877 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:33.877 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:33.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 msgr2=0x7fcdcc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 0x7fcdcc062a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcddc003aa0 tx=0x7fcddc000bc0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 msgr2=0x7fcde80c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c5b0 tx=0x7fcdd4007d30 comp rx=0 tx=0).stop 2023-10-07T23:21:33.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 shutdown_connections 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcdcc060550 0x7fcdcc062a00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcde8159130 0x7fcde80c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcde8158d60 0x7fcde80c4af0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 --2- 172.21.15.57:0/2130591792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcde806bc90 0x7fcde80adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 >> 172.21.15.57:0/2130591792 conn(0x7fcde80670e0 msgr2=0x7fcde806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 shutdown_connections 2023-10-07T23:21:33.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.874+0000 7fcdedf07700 1 -- 172.21.15.57:0/2130591792 wait complete. 2023-10-07T23:21:33.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:33.890 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:33.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 Processor -- start 2023-10-07T23:21:33.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 -- start start 2023-10-07T23:21:33.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 0x7f5b0c15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 0x7f5b0c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:33.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c06c800 con 0x7f5b0c1590b0 2023-10-07T23:21:33.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c06cdc0 con 0x7f5b0c158ce0 2023-10-07T23:21:33.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b11046700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0ae850 con 0x7f5b0c15d080 2023-10-07T23:21:33.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b10845700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5b0c1590b0 0x7f5b0c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60572/0 (socket says 172.21.15.57:60572) 2023-10-07T23:21:33.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b10845700 1 -- 172.21.15.57:0/59056650 learned_addr learned my addr 172.21.15.57:0/59056650 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:33.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b0b7fe700 1 --2- 172.21.15.57:0/59056650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 0x7f5b0c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.990+0000 7f5b0bfff700 1 --2- 172.21.15.57:0/59056650 >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 0x7f5b0c15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:33.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0b7fe700 1 -- 172.21.15.57:0/59056650 >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 msgr2=0x7f5b0c15cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:33.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0b7fe700 1 --2- 172.21.15.57:0/59056650 >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 0x7f5b0c15cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:33.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0b7fe700 1 -- 172.21.15.57:0/59056650 >> v1:172.21.15.57:6789/0 conn(0x7f5b0c1590b0 legacy=0x7f5b0c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:33.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0b7fe700 1 -- 172.21.15.57:0/59056650 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c15d600 con 0x7f5b0c15d080 2023-10-07T23:21:33.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0bfff700 1 --2- 172.21.15.57:0/59056650 >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 0x7f5b0c15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:33.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0b7fe700 1 --2- 172.21.15.57:0/59056650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 0x7f5b0c0c93c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b000098e0 tx=0x7f5b000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cb46d1800365c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0affd700 1 -- 172.21.15.57:0/59056650 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000b640 con 0x7f5b0c15d080 2023-10-07T23:21:34.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0affd700 1 -- 172.21.15.57:0/59056650 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0000b7a0 con 0x7f5b0c15d080 2023-10-07T23:21:34.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.994+0000 7f5b0affd700 1 -- 172.21.15.57:0/59056650 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000bb40 con 0x7f5b0c15d080 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- 172.21.15.57:0/59056650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 msgr2=0x7f5b0c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 --2- 172.21.15.57:0/59056650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 0x7f5b0c0c93c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b000098e0 tx=0x7f5b000091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- 172.21.15.57:0/59056650 shutdown_connections 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 --2- 172.21.15.57:0/59056650 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c15d080 0x7f5b0c0c93c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 --2- 172.21.15.57:0/59056650 >> v2:172.21.15.154:3300/0 conn(0x7f5b0c158ce0 0x7f5b0c15cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- 172.21.15.57:0/59056650 >> 172.21.15.57:0/59056650 conn(0x7f5b0c067b40 msgr2=0x7f5b0c0c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- 172.21.15.57:0/59056650 shutdown_connections 2023-10-07T23:21:34.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- 172.21.15.57:0/59056650 wait complete. 2023-10-07T23:21:34.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 Processor -- start 2023-10-07T23:21:34.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:33.998+0000 7f5b11046700 1 -- start start 2023-10-07T23:21:34.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c158ce0 0x7f5b0c0b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b0c1590b0 0x7f5b0c143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0b0940 con 0x7f5b0c15d080 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0af360 con 0x7f5b0c1590b0 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0af640 con 0x7f5b0c158ce0 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58684/0 (socket says 172.21.15.57:58684) 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 -- 172.21.15.57:0/3162244402 learned_addr learned my addr 172.21.15.57:0/3162244402 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b0bfff700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c158ce0 0x7f5b0c0b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b0b7fe700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b0c1590b0 0x7f5b0c143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 -- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c158ce0 msgr2=0x7f5b0c0b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c158ce0 0x7f5b0c0b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 -- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b0c1590b0 msgr2=0x7f5b0c143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b0c1590b0 0x7f5b0c143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 -- 172.21.15.57:0/3162244402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009420 con 0x7f5b0c15d080 2023-10-07T23:21:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b10845700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400cc40 tx=0x7f5b04001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400a8b0 con 0x7f5b0c15d080 2023-10-07T23:21:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0400aa10 con 0x7f5b0c15d080 2023-10-07T23:21:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b00009420 con 0x7f5b0c15d080 2023-10-07T23:21:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400add0 con 0x7f5b0c15d080 2023-10-07T23:21:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.002+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c0afc00 con 0x7f5b0c15d080 2023-10-07T23:21:34.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0711c0 con 0x7f5b0c15d080 2023-10-07T23:21:34.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5b04007300 con 0x7f5b0c15d080 2023-10-07T23:21:34.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5af3fff700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 0x7f5af8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b0407cb80 con 0x7f5b0c15d080 2023-10-07T23:21:34.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5b0bfff700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 0x7f5af8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.006+0000 7f5b0bfff700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 0x7f5af8062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c15f5b0 tx=0x7f5afc0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.014+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5b0404d0d0 con 0x7f5b0c15d080 2023-10-07T23:21:34.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b0c063030 con 0x7f5b0c15d080 2023-10-07T23:21:34.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5af3fff700 1 -- 172.21.15.57:0/3162244402 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b0407ce60 con 0x7f5b0c15d080 2023-10-07T23:21:34.180 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 msgr2=0x7f5af8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 0x7f5af8062a00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c15f5b0 tx=0x7f5afc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 msgr2=0x7f5b0c15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.174+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400cc40 tx=0x7f5b04001a50 comp rx=0 tx=0).stop 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 shutdown_connections 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b0c15d080 0x7f5b0c15b180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5af8060550 0x7f5af8062a00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b0c1590b0 0x7f5b0c143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 --2- 172.21.15.57:0/3162244402 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5b0c158ce0 0x7f5b0c0b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 >> 172.21.15.57:0/3162244402 conn(0x7f5b0c067b40 msgr2=0x7f5b0c068010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 shutdown_connections 2023-10-07T23:21:34.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.178+0000 7f5b11046700 1 -- 172.21.15.57:0/3162244402 wait complete. 2023-10-07T23:21:34.194 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.298+0000 7f84dd08a700 1 Processor -- start 2023-10-07T23:21:34.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.298+0000 7f84dd08a700 1 -- start start 2023-10-07T23:21:34.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dd08a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 0x7f84d8159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dd08a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84d806a6f0 0x7f84d806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dd08a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80ca530 con 0x7f84d815d580 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dd08a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d815d9f0 con 0x7f84d806a6f0 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dd08a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d806ae60 con 0x7f84d8158c40 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84cffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f84d815d580 0x7f84d80afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60578/0 (socket says 172.21.15.57:60578) 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84cffff700 1 -- 172.21.15.57:0/2120416504 learned_addr learned my addr 172.21.15.57:0/2120416504 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d7fff700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 0x7f84d8159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 --2- 172.21.15.57:0/2120416504 >> v2:172.21.15.154:3300/0 conn(0x7f84d806a6f0 0x7f84d806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190646049 0 0) 0x7f84d80ca530 con 0x7f84d815d580 2023-10-07T23:21:34.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b8003570 con 0x7f84d815d580 2023-10-07T23:21:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 -- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 msgr2=0x7f84d8159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 0x7f84d8159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 -- 172.21.15.57:0/2120416504 >> v1:172.21.15.57:6789/0 conn(0x7f84d815d580 legacy=0x7f84d80afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 251920189 0 0) 0x7f84b8003570 con 0x7f84d815d580 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 -- 172.21.15.57:0/2120416504 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d806b140 con 0x7f84d806a6f0 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d7fff700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 0x7f84d8159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84dc889700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d806a6f0 0x7f84d806c800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e60 tx=0x7f84d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1049dd9a81f43d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000e070 con 0x7f84d806a6f0 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d000b7a0 con 0x7f84d806a6f0 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.302+0000 7f84d77fe700 1 -- 172.21.15.57:0/2120416504 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0001470 con 0x7f84d806a6f0 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d806a6f0 msgr2=0x7f84d806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d806a6f0 0x7f84d806c800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009e60 tx=0x7f84d0009780 comp rx=0 tx=0).stop 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- 172.21.15.57:0/2120416504 shutdown_connections 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d806a6f0 0x7f84d806c800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2120416504 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d8158c40 0x7f84d8159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- 172.21.15.57:0/2120416504 >> 172.21.15.57:0/2120416504 conn(0x7f84d8062ba0 msgr2=0x7f84d8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- 172.21.15.57:0/2120416504 shutdown_connections 2023-10-07T23:21:34.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- 172.21.15.57:0/2120416504 wait complete. 2023-10-07T23:21:34.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 Processor -- start 2023-10-07T23:21:34.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.306+0000 7f84dd08a700 1 -- start start 2023-10-07T23:21:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d806a6f0 0x7f84d80c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 0x7f84d80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d8153740 con 0x7f84d8158c40 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d815a470 con 0x7f84d815d580 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d815aa30 con 0x7f84d806a6f0 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84cffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84cffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58686/0 (socket says 172.21.15.57:58686) 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84cffff700 1 -- 172.21.15.57:0/2536984186 learned_addr learned my addr 172.21.15.57:0/2536984186 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 0x7f84d80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84d7fff700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d806a6f0 0x7f84d80c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 -- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d806a6f0 msgr2=0x7f84d80c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d806a6f0 0x7f84d80c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 -- 172.21.15.57:0/2536984186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 msgr2=0x7f84d80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 -- 172.21.15.57:0/2536984186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8009420 con 0x7f84d815d580 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84cffff700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dc889700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 0x7f84d80b0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f84d806be50 tx=0x7f84d0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0012b80 con 0x7f84d815d580 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d0012ce0 con 0x7f84d815d580 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0009a20 con 0x7f84d815d580 2023-10-07T23:21:34.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d00276c0 con 0x7f84d815d580 2023-10-07T23:21:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d80c6180 con 0x7f84d815d580 2023-10-07T23:21:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.310+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d815abc0 con 0x7f84d815d580 2023-10-07T23:21:34.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.314+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f84d0024070 con 0x7f84d815d580 2023-10-07T23:21:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.314+0000 7f84d4ff9700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 0x7f84bc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.314+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f84d007f3d0 con 0x7f84d815d580 2023-10-07T23:21:34.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.314+0000 7f84d7fff700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 0x7f84bc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.314+0000 7f84d7fff700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 0x7f84bc062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009f40 tx=0x7f84c80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.322+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f84d004fa40 con 0x7f84d815d580 2023-10-07T23:21:34.490 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.486+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84d806b420 con 0x7f84d815d580 2023-10-07T23:21:34.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.486+0000 7f84d4ff9700 1 -- 172.21.15.57:0/2536984186 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84d0053060 con 0x7f84d815d580 2023-10-07T23:21:34.491 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:34.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 msgr2=0x7f84bc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 0x7f84bc062a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009f40 tx=0x7f84c80091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 msgr2=0x7f84d80b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 0x7f84d80b0a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f84d806be50 tx=0x7f84d0018000 comp rx=0 tx=0).stop 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 shutdown_connections 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f84bc060550 0x7f84bc062a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84d815d580 0x7f84d80b0a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f84d8158c40 0x7f84d80c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 --2- 172.21.15.57:0/2536984186 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f84d806a6f0 0x7f84d80c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 >> 172.21.15.57:0/2536984186 conn(0x7f84d8062ba0 msgr2=0x7f84d80d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 shutdown_connections 2023-10-07T23:21:34.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.490+0000 7f84dd08a700 1 -- 172.21.15.57:0/2536984186 wait complete. 2023-10-07T23:21:34.510 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 Processor -- start 2023-10-07T23:21:34.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 -- start start 2023-10-07T23:21:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ee80cad10 0x7f1ee80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee806a900 con 0x7f1ee8159030 2023-10-07T23:21:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee806a600 con 0x7f1ee806cf10 2023-10-07T23:21:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1eee1c6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80b0450 con 0x7f1ee80cad10 2023-10-07T23:21:34.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee77fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f1ee8159030 0x7f1ee806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60588/0 (socket says 172.21.15.57:60588) 2023-10-07T23:21:34.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee77fe700 1 -- 172.21.15.57:0/3928060082 learned_addr learned my addr 172.21.15.57:0/3928060082 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:34.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/3928060082 >> v2:172.21.15.154:3300/0 conn(0x7f1ee80cad10 0x7f1ee80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080594769 0 0) 0x7f1ee806a900 con 0x7f1ee8159030 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed4003570 con 0x7f1ee8159030 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611789440 0 0) 0x7f1ed4003570 con 0x7f1ee8159030 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 >> v2:172.21.15.154:3300/0 conn(0x7f1ee80cad10 msgr2=0x7f1ee80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 --2- 172.21.15.57:0/3928060082 >> v2:172.21.15.154:3300/0 conn(0x7f1ee80cad10 0x7f1ee80aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 >> v1:172.21.15.154:6789/0 conn(0x7f1ee806cf10 legacy=0x7f1ee80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80cb350 con 0x7f1ee8159030 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f1ed8001470 con 0x7f1ee8159030 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 >> v1:172.21.15.57:6789/0 conn(0x7f1ee8159030 legacy=0x7f1ee806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ed4003a60 0x7f1ed4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80b0450 con 0x7f1ed40051c0 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400c0d0 con 0x7f1ed4003a60 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed400c290 con 0x7f1ed4004810 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.618+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1edeffd700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee77fe700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ed4003a60 0x7f1ed4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 -- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed4004810 msgr2=0x7f1ed4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed4004810 0x7f1ed4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 -- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ed4003a60 msgr2=0x7f1ed4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ed4003a60 0x7f1ed4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 -- 172.21.15.57:0/3928060082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed4003570 con 0x7f1ed40051c0 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 -- 172.21.15.57:0/3928060082 reap_dead start 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e210 tx=0x7f1ee000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107733cec1f83961 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000eac0 con 0x7f1ed40051c0 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000f040 con 0x7f1ed40051c0 2023-10-07T23:21:34.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1ee6ffd700 1 -- 172.21.15.57:0/3928060082 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000ee70 con 0x7f1ed40051c0 2023-10-07T23:21:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 msgr2=0x7f1ed400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e210 tx=0x7f1ee000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- 172.21.15.57:0/3928060082 shutdown_connections 2023-10-07T23:21:34.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/3928060082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ed40051c0 0x7f1ed400ba60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- 172.21.15.57:0/3928060082 >> 172.21.15.57:0/3928060082 conn(0x7f1ee8068650 msgr2=0x7f1ee8068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- 172.21.15.57:0/3928060082 shutdown_connections 2023-10-07T23:21:34.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- 172.21.15.57:0/3928060082 wait complete. 2023-10-07T23:21:34.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 Processor -- start 2023-10-07T23:21:34.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.622+0000 7f1eee1c6700 1 -- start start 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 0x7f1ee81536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ee80c5dc0 0x7f1ee80c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80b2da0 con 0x7f1ee80b3db0 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80b3920 con 0x7f1ee80c5dc0 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80b2ff0 con 0x7f1ed40051c0 2023-10-07T23:21:34.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee77fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 0x7f1ee81536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58704/0 (socket says 172.21.15.57:58704) 2023-10-07T23:21:34.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee77fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 0x7f1ee81536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:57204/0 (socket says 172.21.15.57:57204) 2023-10-07T23:21:34.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee77fe700 1 -- 172.21.15.57:0/30109876 learned_addr learned my addr 172.21.15.57:0/30109876 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:34.633 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1edeffd700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ee80c5dc0 0x7f1ee80c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 -- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 msgr2=0x7f1ee81536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 0x7f1ee81536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 -- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ee80c5dc0 msgr2=0x7f1ee80c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ee80c5dc0 0x7f1ee80c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 -- 172.21.15.57:0/30109876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80b32d0 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1ee7fff700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ee806a460 tx=0x7f1ee0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001d660 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0022070 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee80b35b0 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001da30 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.626+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee80c6750 con 0x7f1ee80b3db0 2023-10-07T23:21:34.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.630+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee80b3130 con 0x7f1ee80b3db0 2023-10-07T23:21:34.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.630+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1ee0003020 con 0x7f1ee80b3db0 2023-10-07T23:21:34.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.630+0000 7f1edffff700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 0x7f1ec8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.630+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ee0024070 con 0x7f1ee80b3db0 2023-10-07T23:21:34.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.630+0000 7f1ee77fe700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 0x7f1ec8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.634+0000 7f1ee77fe700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 0x7f1ec8062a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001ed0 tx=0x7f1ed8007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.638+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1ee80b3130 con 0x7f1ee80b3db0 2023-10-07T23:21:34.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.798+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ee80b3130 con 0x7f1ee80b3db0 2023-10-07T23:21:34.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1edffff700 1 -- 172.21.15.57:0/30109876 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ee80b3130 con 0x7f1ee80b3db0 2023-10-07T23:21:34.807 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 msgr2=0x7f1ec8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 0x7f1ec8062a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001ed0 tx=0x7f1ed8007040 comp rx=0 tx=0).stop 2023-10-07T23:21:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 msgr2=0x7f1ee80b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ee806a460 tx=0x7f1ee0019000 comp rx=0 tx=0).stop 2023-10-07T23:21:34.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 shutdown_connections 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1ee80b3db0 0x7f1ee80b4200 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1ec8060580 0x7f1ec8062a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ee80c5dc0 0x7f1ee80c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 --2- 172.21.15.57:0/30109876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1ed40051c0 0x7f1ee81536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 >> 172.21.15.57:0/30109876 conn(0x7f1ee8068650 msgr2=0x7f1ee815cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 shutdown_connections 2023-10-07T23:21:34.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.802+0000 7f1eee1c6700 1 -- 172.21.15.57:0/30109876 wait complete. 2023-10-07T23:21:34.820 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.922+0000 7fcb7d444700 1 Processor -- start 2023-10-07T23:21:34.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.922+0000 7fcb7d444700 1 -- start start 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 0x7fcb781596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb78150290 0x7fcb7806b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb7806d810 0x7fcb7806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb78067590 con 0x7fcb7806d810 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7806e1d0 con 0x7fcb7814fec0 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7d444700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780cb2e0 con 0x7fcb78150290 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb7806d810 0x7fcb7806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcb7806d810 0x7fcb7806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58714/0 (socket says 172.21.15.57:58714) 2023-10-07T23:21:34.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 -- 172.21.15.57:0/2522116308 learned_addr learned my addr 172.21.15.57:0/2522116308 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:34.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb77fff700 1 --2- 172.21.15.57:0/2522116308 >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 0x7fcb781596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb777fe700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb78150290 0x7fcb7806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 -- 172.21.15.57:0/2522116308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb78150290 msgr2=0x7fcb7806b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb78150290 0x7fcb7806b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 -- 172.21.15.57:0/2522116308 >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 msgr2=0x7fcb781596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:34.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 --2- 172.21.15.57:0/2522116308 >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 0x7fcb781596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 -- 172.21.15.57:0/2522116308 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780cb5c0 con 0x7fcb7806d810 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb77fff700 1 --2- 172.21.15.57:0/2522116308 >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 0x7fcb781596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb7cc43700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7806d810 0x7fcb7806dc90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb780c9c60 tx=0x7fcb70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f905b01de664b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb76ffd700 1 -- 172.21.15.57:0/2522116308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7000e070 con 0x7fcb7806d810 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb76ffd700 1 -- 172.21.15.57:0/2522116308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7000b970 con 0x7fcb7806d810 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.926+0000 7fcb76ffd700 1 -- 172.21.15.57:0/2522116308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb700092e0 con 0x7fcb7806d810 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- 172.21.15.57:0/2522116308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7806d810 msgr2=0x7fcb7806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7806d810 0x7fcb7806dc90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb780c9c60 tx=0x7fcb70009840 comp rx=0 tx=0).stop 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- 172.21.15.57:0/2522116308 shutdown_connections 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7806d810 0x7fcb7806dc90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- 172.21.15.57:0/2522116308 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb78150290 0x7fcb7806b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- 172.21.15.57:0/2522116308 >> v2:172.21.15.154:3300/0 conn(0x7fcb7814fec0 0x7fcb781596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- 172.21.15.57:0/2522116308 >> 172.21.15.57:0/2522116308 conn(0x7fcb780c9750 msgr2=0x7fcb7806acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:34.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- 172.21.15.57:0/2522116308 shutdown_connections 2023-10-07T23:21:34.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- 172.21.15.57:0/2522116308 wait complete. 2023-10-07T23:21:34.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 Processor -- start 2023-10-07T23:21:34.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- start start 2023-10-07T23:21:34.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 0x7fcb780b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 0x7fcb780c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780c3c20 con 0x7fcb7814fec0 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780b13b0 con 0x7fcb78150290 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb7d444700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780c32f0 con 0x7fcb7806d810 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb777fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb777fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58730/0 (socket says 172.21.15.57:58730) 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb777fe700 1 -- 172.21.15.57:0/494888475 learned_addr learned my addr 172.21.15.57:0/494888475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.930+0000 7fcb77fff700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 0x7fcb780b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb7cc43700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 0x7fcb780c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 -- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 msgr2=0x7fcb780b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 0x7fcb780b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 -- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 msgr2=0x7fcb780c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 0x7fcb780c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 -- 172.21.15.57:0/494888475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009420 con 0x7fcb7814fec0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb7cc43700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 0x7fcb780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb77fff700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 0x7fcb780b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb777fe700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e900 tx=0x7fcb6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00eea0 con 0x7fcb7814fec0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00c850 con 0x7fcb7814fec0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb70009ae0 con 0x7fcb7814fec0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00cc10 con 0x7fcb7814fec0 2023-10-07T23:21:34.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb780c38b0 con 0x7fcb7814fec0 2023-10-07T23:21:34.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb780711c0 con 0x7fcb7814fec0 2023-10-07T23:21:34.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcb6c013470 con 0x7fcb7814fec0 2023-10-07T23:21:34.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.934+0000 7fcb5ffff700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 0x7fcb64062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:34.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.938+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb6c07c6d0 con 0x7fcb7814fec0 2023-10-07T23:21:34.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.938+0000 7fcb77fff700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 0x7fcb64062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:34.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.938+0000 7fcb77fff700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 0x7fcb64062a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb680093f0 tx=0x7fcb68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:34.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:34.942+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcb6c04ccc0 con 0x7fcb7814fec0 2023-10-07T23:21:35.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.138+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb780cb8a0 con 0x7fcb7814fec0 2023-10-07T23:21:35.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.138+0000 7fcb5ffff700 1 -- 172.21.15.57:0/494888475 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fcb6c0502e0 con 0x7fcb7814fec0 2023-10-07T23:21:35.144 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:35.144 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 msgr2=0x7fcb64062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 0x7fcb64062a30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcb680093f0 tx=0x7fcb68008040 comp rx=0 tx=0).stop 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 msgr2=0x7fcb7814bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00e900 tx=0x7fcb6c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 shutdown_connections 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcb64060580 0x7fcb64062a30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb78150290 0x7fcb780c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcb7814fec0 0x7fcb7814bc40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 --2- 172.21.15.57:0/494888475 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcb7806d810 0x7fcb780b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 >> 172.21.15.57:0/494888475 conn(0x7fcb780c9750 msgr2=0x7fcb7814dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 shutdown_connections 2023-10-07T23:21:35.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.142+0000 7fcb7d444700 1 -- 172.21.15.57:0/494888475 wait complete. 2023-10-07T23:21:35.148 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:35.159 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 Processor -- start 2023-10-07T23:21:35.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 -- start start 2023-10-07T23:21:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152af0 0x7f608c156520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f608c0ca220 0x7f608c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c06b2f0 con 0x7f608c152720 2023-10-07T23:21:35.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c1598f0 con 0x7f608c0ca220 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f6093050700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0cab90 con 0x7f608c152af0 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f608c0ca220 0x7f608c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609204e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f608c152720 0x7f608c158950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60602/0 (socket says 172.21.15.57:60602) 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f608c0ca220 0x7f608c0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58740/0 (socket says 172.21.15.57:58740) 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609204e700 1 -- 172.21.15.57:0/4286663641 learned_addr learned my addr 172.21.15.57:0/4286663641 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609184d700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152af0 0x7f608c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 -- 172.21.15.57:0/4286663641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152af0 msgr2=0x7f608c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:35.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152af0 0x7f608c156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 -- 172.21.15.57:0/4286663641 >> v1:172.21.15.57:6789/0 conn(0x7f608c152720 legacy=0x7f608c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 -- 172.21.15.57:0/4286663641 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c159060 con 0x7f608c0ca220 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609284f700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0ca650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098e0 tx=0x7f607c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80ff89f00da98aa server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609104c700 1 -- 172.21.15.57:0/4286663641 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c009e80 con 0x7f608c0ca220 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609104c700 1 -- 172.21.15.57:0/4286663641 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c00b860 con 0x7f608c0ca220 2023-10-07T23:21:35.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.258+0000 7f609104c700 1 -- 172.21.15.57:0/4286663641 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c00bbe0 con 0x7f608c0ca220 2023-10-07T23:21:35.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 -- 172.21.15.57:0/4286663641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 msgr2=0x7f608c0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0ca650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f607c0098e0 tx=0x7f607c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 -- 172.21.15.57:0/4286663641 shutdown_connections 2023-10-07T23:21:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152af0 0x7f608c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 --2- 172.21.15.57:0/4286663641 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0ca650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 -- 172.21.15.57:0/4286663641 >> 172.21.15.57:0/4286663641 conn(0x7f608c0670e0 msgr2=0x7f608c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 -- 172.21.15.57:0/4286663641 shutdown_connections 2023-10-07T23:21:35.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.262+0000 7f6093050700 1 -- 172.21.15.57:0/4286663641 wait complete. 2023-10-07T23:21:35.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 Processor -- start 2023-10-07T23:21:35.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- start start 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152720 0x7f608c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f608c152af0 0x7f608c13c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c2a40 con 0x7f608c0ca220 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c2d20 con 0x7f608c152af0 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c3000 con 0x7f608c152720 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58742/0 (socket says 172.21.15.57:58742) 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 -- 172.21.15.57:0/1143052683 learned_addr learned my addr 172.21.15.57:0/1143052683 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:35.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609184d700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152720 0x7f608c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609284f700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f608c152af0 0x7f608c13c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 -- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152720 msgr2=0x7f608c0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152720 0x7f608c0c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 -- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f608c152af0 msgr2=0x7f608c13c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f608c152af0 0x7f608c13c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 -- 172.21.15.57:0/1143052683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c009420 con 0x7f608c0ca220 2023-10-07T23:21:35.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f609204e700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f608400cd30 tx=0x7f6084001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400ce30 con 0x7f608c0ca220 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0c32e0 con 0x7f608c0ca220 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084004bb0 con 0x7f608c0ca220 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400aac0 con 0x7f608c0ca220 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.266+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c151f30 con 0x7f608c0ca220 2023-10-07T23:21:35.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.270+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c0c37e0 con 0x7f608c0ca220 2023-10-07T23:21:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.270+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f60840070f0 con 0x7f608c0ca220 2023-10-07T23:21:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.270+0000 7f608a7fc700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 0x7f6078062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.270+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f608407c110 con 0x7f608c0ca220 2023-10-07T23:21:35.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.270+0000 7f609184d700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 0x7f6078062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.274+0000 7f609184d700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 0x7f6078062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f608c14a1d0 tx=0x7f6080008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.278+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f608404c700 con 0x7f608c0ca220 2023-10-07T23:21:35.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.434+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f608c0711c0 con 0x7f608c0ca220 2023-10-07T23:21:35.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.438+0000 7f608a7fc700 1 -- 172.21.15.57:0/1143052683 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f608404fd20 con 0x7f608c0ca220 2023-10-07T23:21:35.444 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:35.444 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:35.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 msgr2=0x7f6078062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 0x7f6078062a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f608c14a1d0 tx=0x7f6080008040 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 msgr2=0x7f608c0c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f608400cd30 tx=0x7f6084001ad0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 shutdown_connections 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6078060550 0x7f6078062a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f608c152af0 0x7f608c13c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f608c152720 0x7f608c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 --2- 172.21.15.57:0/1143052683 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f608c0ca220 0x7f608c0c3ae0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 >> 172.21.15.57:0/1143052683 conn(0x7f608c0670e0 msgr2=0x7f608c157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 shutdown_connections 2023-10-07T23:21:35.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.442+0000 7f6093050700 1 -- 172.21.15.57:0/1143052683 wait complete. 2023-10-07T23:21:35.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:35.459 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:35.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.570+0000 7fb1da881700 1 Processor -- start 2023-10-07T23:21:35.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.570+0000 7fb1da881700 1 -- start start 2023-10-07T23:21:35.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1d40c73f0 0x7fb1d40693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d4069900 con 0x7fb1d4068a10 2023-10-07T23:21:35.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d40c7870 con 0x7fb1d41554f0 2023-10-07T23:21:35.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d40aee70 con 0x7fb1d40c73f0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1d40c73f0 0x7fb1d40693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1d40c73f0 0x7fb1d40693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58756/0 (socket says 172.21.15.57:58756) 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 -- 172.21.15.57:0/3969036363 learned_addr learned my addr 172.21.15.57:0/3969036363 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 -- 172.21.15.57:0/3969036363 >> v1:172.21.15.154:6789/0 conn(0x7fb1d41554f0 legacy=0x7fb1d4068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 -- 172.21.15.57:0/3969036363 >> v1:172.21.15.57:6789/0 conn(0x7fb1d4068a10 legacy=0x7fb1d40ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 -- 172.21.15.57:0/3969036363 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40acb70 con 0x7fb1d40c73f0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d8e1e700 1 --2- 172.21.15.57:0/3969036363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d40c73f0 0x7fb1d40693c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c0d0 tx=0x7fb1cc00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4048d519db1058 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d37fe700 1 -- 172.21.15.57:0/3969036363 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00f070 con 0x7fb1d40c73f0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d37fe700 1 -- 172.21.15.57:0/3969036363 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc00a2e0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1d37fe700 1 -- 172.21.15.57:0/3969036363 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00cdf0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- 172.21.15.57:0/3969036363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d40c73f0 msgr2=0x7fb1d40693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 --2- 172.21.15.57:0/3969036363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d40c73f0 0x7fb1d40693c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c0d0 tx=0x7fb1cc00a950 comp rx=0 tx=0).stop 2023-10-07T23:21:35.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- 172.21.15.57:0/3969036363 shutdown_connections 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 --2- 172.21.15.57:0/3969036363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d40c73f0 0x7fb1d40693c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- 172.21.15.57:0/3969036363 >> 172.21.15.57:0/3969036363 conn(0x7fb1d40ccf10 msgr2=0x7fb1d40c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- 172.21.15.57:0/3969036363 shutdown_connections 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.574+0000 7fb1da881700 1 -- 172.21.15.57:0/3969036363 wait complete. 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 Processor -- start 2023-10-07T23:21:35.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 -- start start 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 0x7fb1d40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 0x7fb1d4145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 0x7fb1d4145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41682a0 con 0x7fb1d4068a10 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4168580 con 0x7fb1d40c73f0 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1da881700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4168860 con 0x7fb1d41554f0 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 0x7fb1d40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 0x7fb1d40ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58766/0 (socket says 172.21.15.57:58766) 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1cbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 0x7fb1d4145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1d8e1e700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 0x7fb1d4145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.578+0000 7fb1d3fff700 1 -- 172.21.15.57:0/3294965830 learned_addr learned my addr 172.21.15.57:0/3294965830 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 -- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 msgr2=0x7fb1d4145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 0x7fb1d4145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 -- 172.21.15.57:0/3294965830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 msgr2=0x7fb1d40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 0x7fb1d40ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 -- 172.21.15.57:0/3294965830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc00abf0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1d8e1e700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 0x7fb1d4145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1cbfff700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 0x7fb1d4145400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c560 tx=0x7fb1bc007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc011590 con 0x7fb1d40c73f0 2023-10-07T23:21:35.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc011710 con 0x7fb1d40c73f0 2023-10-07T23:21:35.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d4168b40 con 0x7fb1d40c73f0 2023-10-07T23:21:35.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc011ad0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d40ca910 con 0x7fb1d40c73f0 2023-10-07T23:21:35.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.582+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d4168d30 con 0x7fb1d40c73f0 2023-10-07T23:21:35.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.586+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb1bc01b3b0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.586+0000 7fb1d0ff9700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 0x7fb1b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.586+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1bc07bae0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.586+0000 7fb1d3fff700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 0x7fb1b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.586+0000 7fb1d3fff700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 0x7fb1b4062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40078a0 tx=0x7fb1c4007830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.598+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb1bc04c0d0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.758+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1d41686c0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.763 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.758+0000 7fb1d0ff9700 1 -- 172.21.15.57:0/3294965830 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1d41686c0 con 0x7fb1d40c73f0 2023-10-07T23:21:35.763 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:35.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 msgr2=0x7fb1b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 0x7fb1b4062a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40078a0 tx=0x7fb1c4007830 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 msgr2=0x7fb1d4145400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 0x7fb1d4145400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c560 tx=0x7fb1bc007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 shutdown_connections 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1d41554f0 0x7fb1d4145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1b4060550 0x7fb1b4062a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1d40c73f0 0x7fb1d4145400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 --2- 172.21.15.57:0/3294965830 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1d4068a10 0x7fb1d40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 >> 172.21.15.57:0/3294965830 conn(0x7fb1d40ccf10 msgr2=0x7fb1d40b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 shutdown_connections 2023-10-07T23:21:35.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.762+0000 7fb1da881700 1 -- 172.21.15.57:0/3294965830 wait complete. 2023-10-07T23:21:35.784 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:35.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.890+0000 7ff3a5ae7700 1 Processor -- start 2023-10-07T23:21:35.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.890+0000 7ff3a5ae7700 1 -- start start 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3a0158ce0 0x7ff3a015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff3a015d080 0x7ff3a00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a006c800 con 0x7ff3a01590b0 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a006cdc0 con 0x7ff3a015d080 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00ae850 con 0x7ff3a0158ce0 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff3a015d080 0x7ff3a00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a52e6700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff3a01590b0 0x7ff3a006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60612/0 (socket says 172.21.15.57:60612) 2023-10-07T23:21:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a52e6700 1 -- 172.21.15.57:0/2720955750 learned_addr learned my addr 172.21.15.57:0/2720955750 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:35.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a4ae5700 1 --2- 172.21.15.57:0/2720955750 >> v2:172.21.15.154:3300/0 conn(0x7ff3a0158ce0 0x7ff3a015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 -- 172.21.15.57:0/2720955750 >> v2:172.21.15.154:3300/0 conn(0x7ff3a0158ce0 msgr2=0x7ff3a015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 --2- 172.21.15.57:0/2720955750 >> v2:172.21.15.154:3300/0 conn(0x7ff3a0158ce0 0x7ff3a015cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 -- 172.21.15.57:0/2720955750 >> v1:172.21.15.57:6789/0 conn(0x7ff3a01590b0 legacy=0x7ff3a006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3977fe700 1 -- 172.21.15.57:0/2720955750 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651584191 0 0) 0x7ff3a006c800 con 0x7ff3a01590b0 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 -- 172.21.15.57:0/2720955750 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a015d600 con 0x7ff3a015d080 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff397fff700 1 --2- 172.21.15.57:0/2720955750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a015d080 0x7ff3a00c93c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c9f00 tx=0x7ff3900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdab87e79a58212 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3977fe700 1 -- 172.21.15.57:0/2720955750 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390011630 con 0x7ff3a015d080 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3977fe700 1 -- 172.21.15.57:0/2720955750 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390011790 con 0x7ff3a015d080 2023-10-07T23:21:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3977fe700 1 -- 172.21.15.57:0/2720955750 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390011b30 con 0x7ff3a015d080 2023-10-07T23:21:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/2720955750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a015d080 msgr2=0x7ff3a00c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.894+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/2720955750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a015d080 0x7ff3a00c93c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c9f00 tx=0x7ff3900091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/2720955750 shutdown_connections 2023-10-07T23:21:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/2720955750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a015d080 0x7ff3a00c93c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/2720955750 >> v2:172.21.15.154:3300/0 conn(0x7ff3a0158ce0 0x7ff3a015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/2720955750 >> 172.21.15.57:0/2720955750 conn(0x7ff3a0067b40 msgr2=0x7ff3a00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/2720955750 shutdown_connections 2023-10-07T23:21:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/2720955750 wait complete. 2023-10-07T23:21:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 Processor -- start 2023-10-07T23:21:35.904 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.898+0000 7ff3a5ae7700 1 -- start start 2023-10-07T23:21:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a01590b0 0x7ff3a015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a015d080 0x7ff3a015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0154390 con 0x7ff3a0158ce0 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0159710 con 0x7ff3a015d080 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01599f0 con 0x7ff3a01590b0 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58784/0 (socket says 172.21.15.57:58784) 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 -- 172.21.15.57:0/1242756897 learned_addr learned my addr 172.21.15.57:0/1242756897 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a52e6700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a015d080 0x7ff3a015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff397fff700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a01590b0 0x7ff3a015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 -- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a01590b0 msgr2=0x7ff3a015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a01590b0 0x7ff3a015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 -- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a015d080 msgr2=0x7ff3a015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a015d080 0x7ff3a015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 -- 172.21.15.57:0/1242756897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390009420 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a4ae5700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e930 tx=0x7ff39c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c013680 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c0137e0 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a0159cd0 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c013b80 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.902+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a015a4d0 con 0x7ff3a0158ce0 2023-10-07T23:21:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a015a150 con 0x7ff3a0158ce0 2023-10-07T23:21:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff39c00c3e0 con 0x7ff3a0158ce0 2023-10-07T23:21:35.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff394ff9700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 0x7ff38c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff39c07c8e0 con 0x7ff3a0158ce0 2023-10-07T23:21:35.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff397fff700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 0x7ff38c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:35.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.906+0000 7ff397fff700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 0x7ff38c062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3900091e0 tx=0x7ff390017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:35.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:35.914+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff39c04ced0 con 0x7ff3a0158ce0 2023-10-07T23:21:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.078+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3a00711c0 con 0x7ff3a0158ce0 2023-10-07T23:21:36.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.078+0000 7ff394ff9700 1 -- 172.21.15.57:0/1242756897 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff39c0504f0 con 0x7ff3a0158ce0 2023-10-07T23:21:36.085 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:36.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 msgr2=0x7ff38c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 0x7ff38c062a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3900091e0 tx=0x7ff390017000 comp rx=0 tx=0).stop 2023-10-07T23:21:36.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 msgr2=0x7ff3a01531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00e930 tx=0x7ff39c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 shutdown_connections 2023-10-07T23:21:36.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff38c060550 0x7ff38c062a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a015d080 0x7ff3a015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff3a01590b0 0x7ff3a015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 --2- 172.21.15.57:0/1242756897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff3a0158ce0 0x7ff3a01531f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 >> 172.21.15.57:0/1242756897 conn(0x7ff3a0067b40 msgr2=0x7ff3a00c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 shutdown_connections 2023-10-07T23:21:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.082+0000 7ff3a5ae7700 1 -- 172.21.15.57:0/1242756897 wait complete. 2023-10-07T23:21:36.099 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.206+0000 7f744862c700 1 Processor -- start 2023-10-07T23:21:36.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.206+0000 7f744862c700 1 -- start start 2023-10-07T23:21:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 0x7f74400cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f74400cc920 0x7f74400c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440067290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400cab20 con 0x7f74400cc920 2023-10-07T23:21:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400ca260 con 0x7f7440159100 2023-10-07T23:21:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400ccd50 con 0x7f74400c8ed0 2023-10-07T23:21:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f74400cc920 0x7f74400c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f74400cc920 0x7f74400c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58796/0 (socket says 172.21.15.57:58796) 2023-10-07T23:21:36.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 -- 172.21.15.57:0/626418705 learned_addr learned my addr 172.21.15.57:0/626418705 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f74463c8700 1 --2- 172.21.15.57:0/626418705 >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 0x7f74400cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7446bc9700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440067290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 -- 172.21.15.57:0/626418705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 msgr2=0x7f7440067290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:36.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440067290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 -- 172.21.15.57:0/626418705 >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 msgr2=0x7f74400cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 --2- 172.21.15.57:0/626418705 >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 0x7f74400cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 -- 172.21.15.57:0/626418705 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400c9650 con 0x7f74400cc920 2023-10-07T23:21:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f74463c8700 1 --2- 172.21.15.57:0/626418705 >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 0x7f74400cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:36.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f7445bc7700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400cc920 0x7f74400c8990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f744006b990 tx=0x7f743c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04824c8ffb0d2d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f74453c6700 1 -- 172.21.15.57:0/626418705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00b860 con 0x7f74400cc920 2023-10-07T23:21:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f74453c6700 1 -- 172.21.15.57:0/626418705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c00b9c0 con 0x7f74400cc920 2023-10-07T23:21:36.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f74453c6700 1 -- 172.21.15.57:0/626418705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00bd60 con 0x7f74400cc920 2023-10-07T23:21:36.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- 172.21.15.57:0/626418705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400cc920 msgr2=0x7f74400c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400cc920 0x7f74400c8990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f744006b990 tx=0x7f743c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- 172.21.15.57:0/626418705 shutdown_connections 2023-10-07T23:21:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440067290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- 172.21.15.57:0/626418705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400cc920 0x7f74400c8990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 --2- 172.21.15.57:0/626418705 >> v2:172.21.15.154:3300/0 conn(0x7f7440159100 0x7f74400cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- 172.21.15.57:0/626418705 >> 172.21.15.57:0/626418705 conn(0x7f7440069d70 msgr2=0x7f744006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- 172.21.15.57:0/626418705 shutdown_connections 2023-10-07T23:21:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.210+0000 7f744862c700 1 -- 172.21.15.57:0/626418705 wait complete. 2023-10-07T23:21:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 Processor -- start 2023-10-07T23:21:36.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 -- start start 2023-10-07T23:21:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440169030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7440159100 0x7f74400b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744016c680 con 0x7f74400b22e0 2023-10-07T23:21:36.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440169860 con 0x7f7440159100 2023-10-07T23:21:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f744862c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7440169b40 con 0x7f74400c8ed0 2023-10-07T23:21:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58810/0 (socket says 172.21.15.57:58810) 2023-10-07T23:21:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 -- 172.21.15.57:0/1952727994 learned_addr learned my addr 172.21.15.57:0/1952727994 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:36.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7445bc7700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7440159100 0x7f74400b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f74463c8700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 -- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 msgr2=0x7f7440169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.214+0000 7f7446bc9700 1 -- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7440159100 msgr2=0x7f74400b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f7446bc9700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7440159100 0x7f74400b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f7446bc9700 1 -- 172.21.15.57:0/1952727994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0094f0 con 0x7f74400b22e0 2023-10-07T23:21:36.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f7446bc9700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fa0 tx=0x7f7438009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800ed20 con 0x7f74400b22e0 2023-10-07T23:21:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800ee80 con 0x7f74400b22e0 2023-10-07T23:21:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7440169e20 con 0x7f74400b22e0 2023-10-07T23:21:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438014670 con 0x7f74400b22e0 2023-10-07T23:21:36.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7440169570 con 0x7f74400b22e0 2023-10-07T23:21:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744016caf0 con 0x7f74400b22e0 2023-10-07T23:21:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f74380149e0 con 0x7f74400b22e0 2023-10-07T23:21:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.218+0000 7f74327fc700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 0x7f7424062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.222+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f743807cf80 con 0x7f74400b22e0 2023-10-07T23:21:36.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.222+0000 7f74463c8700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 0x7f7424062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.222+0000 7f74463c8700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 0x7f7424062a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7434006f00 tx=0x7f7434008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.226+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74380acbe0 con 0x7f74400b22e0 2023-10-07T23:21:36.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74400647b0 con 0x7f74400b22e0 2023-10-07T23:21:36.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f74327fc700 1 -- 172.21.15.57:0/1952727994 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f743804d570 con 0x7f74400b22e0 2023-10-07T23:21:36.395 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:36.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 msgr2=0x7f7424062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 0x7f7424062a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7434006f00 tx=0x7f7434008040 comp rx=0 tx=0).stop 2023-10-07T23:21:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 msgr2=0x7f744014a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7438009fa0 tx=0x7f7438009900 comp rx=0 tx=0).stop 2023-10-07T23:21:36.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 shutdown_connections 2023-10-07T23:21:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f74400b22e0 0x7f744014a470 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7424060580 0x7f7424062a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7440159100 0x7f74400b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 --2- 172.21.15.57:0/1952727994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f74400c8ed0 0x7f7440169030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 >> 172.21.15.57:0/1952727994 conn(0x7f7440069d70 msgr2=0x7f74401587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 shutdown_connections 2023-10-07T23:21:36.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.390+0000 7f744862c700 1 -- 172.21.15.57:0/1952727994 wait complete. 2023-10-07T23:21:36.410 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:36.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 Processor -- start 2023-10-07T23:21:36.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 -- start start 2023-10-07T23:21:36.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 0x7f7f740cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f740cafc0 con 0x7f7f740c7100 2023-10-07T23:21:36.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f740c8090 con 0x7f7f74140790 2023-10-07T23:21:36.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79848700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740cc7d0 con 0x7f7f741403c0 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f73fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 0x7f7f740cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:44368/0 (socket says 172.21.15.57:44368) 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f79047700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7f740c7100 0x7f7f7406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:55230/0 (socket says 172.21.15.57:55230) 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 -- 172.21.15.57:0/188638177 learned_addr learned my addr 172.21.15.57:0/188638177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f737fe700 1 -- 172.21.15.57:0/188638177 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468240291 0 0) 0x7f7f740cafc0 con 0x7f7f740c7100 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f737fe700 1 -- 172.21.15.57:0/188638177 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003570 con 0x7f7f740c7100 2023-10-07T23:21:36.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 -- 172.21.15.57:0/188638177 >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 msgr2=0x7f7f740cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:36.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 --2- 172.21.15.57:0/188638177 >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 0x7f7f740cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 -- 172.21.15.57:0/188638177 >> v1:172.21.15.154:6789/0 conn(0x7f7f740c7100 legacy=0x7f7f7406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 -- 172.21.15.57:0/188638177 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740ca690 con 0x7f7f741403c0 2023-10-07T23:21:36.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f73fff700 1 --2- 172.21.15.57:0/188638177 >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 0x7f7f740cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.514+0000 7f7f78846700 1 --2- 172.21.15.57:0/188638177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009f20 tx=0x7f7f64009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c32bf08466ca6ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f737fe700 1 -- 172.21.15.57:0/188638177 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400e070 con 0x7f7f741403c0 2023-10-07T23:21:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f737fe700 1 -- 172.21.15.57:0/188638177 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400b970 con 0x7f7f741403c0 2023-10-07T23:21:36.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f737fe700 1 -- 172.21.15.57:0/188638177 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640092e0 con 0x7f7f741403c0 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- 172.21.15.57:0/188638177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 msgr2=0x7f7f740cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- 172.21.15.57:0/188638177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009f20 tx=0x7f7f64009840 comp rx=0 tx=0).stop 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- 172.21.15.57:0/188638177 shutdown_connections 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- 172.21.15.57:0/188638177 >> v2:172.21.15.154:3300/0 conn(0x7f7f74140790 0x7f7f740cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- 172.21.15.57:0/188638177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f741403c0 0x7f7f740cb450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- 172.21.15.57:0/188638177 >> 172.21.15.57:0/188638177 conn(0x7f7f74095d30 msgr2=0x7f7f7406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- 172.21.15.57:0/188638177 shutdown_connections 2023-10-07T23:21:36.524 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- 172.21.15.57:0/188638177 wait complete. 2023-10-07T23:21:36.525 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 Processor -- start 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- start start 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f741403c0 0x7f7f740c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f74140790 0x7f7f74068290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740c6970 con 0x7f7f740c7100 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7409a160 con 0x7f7f741403c0 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79848700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740c6320 con 0x7f7f74140790 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f78846700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f78846700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34280/0 (socket says 172.21.15.57:34280) 2023-10-07T23:21:36.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f78846700 1 -- 172.21.15.57:0/2097526259 learned_addr learned my addr 172.21.15.57:0/2097526259 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f79047700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f74140790 0x7f7f74068290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f73fff700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f741403c0 0x7f7f740c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.518+0000 7f7f78846700 1 -- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f74140790 msgr2=0x7f7f74068290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f78846700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f74140790 0x7f7f74068290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f78846700 1 -- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f741403c0 msgr2=0x7f7f740c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f78846700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f741403c0 0x7f7f740c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f78846700 1 -- 172.21.15.57:0/2097526259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c009420 con 0x7f7f740c7100 2023-10-07T23:21:36.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f78846700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b610 tx=0x7f7f64018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640088b0 con 0x7f7f740c7100 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f64008a30 con 0x7f7f740c7100 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f64009ae0 con 0x7f7f740c7100 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64008df0 con 0x7f7f740c7100 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f740c6e00 con 0x7f7f740c7100 2023-10-07T23:21:36.528 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f740c6180 con 0x7f7f740c7100 2023-10-07T23:21:36.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.522+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7f64003040 con 0x7f7f740c7100 2023-10-07T23:21:36.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.526+0000 7f7f70ff9700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 0x7f7f58062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.526+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f6407e4c0 con 0x7f7f740c7100 2023-10-07T23:21:36.530 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.526+0000 7f7f73fff700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 0x7f7f58062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.531 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.526+0000 7f7f73fff700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 0x7f7f58062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f7415d150 tx=0x7f7f5c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.530+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f740c6180 con 0x7f7f740c7100 2023-10-07T23:21:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f740c6180 con 0x7f7f740c7100 2023-10-07T23:21:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f70ff9700 1 -- 172.21.15.57:0/2097526259 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f7f740c6180 con 0x7f7f740c7100 2023-10-07T23:21:36.735 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:36.736 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 msgr2=0x7f7f58062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 0x7f7f58062a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7f7415d150 tx=0x7f7f5c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 msgr2=0x7f7f7413e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.730+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b610 tx=0x7f7f64018000 comp rx=0 tx=0).stop 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 shutdown_connections 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f74140790 0x7f7f74068290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f58060550 0x7f7f58062a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f741403c0 0x7f7f740c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 --2- 172.21.15.57:0/2097526259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f740c7100 0x7f7f7413e9e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 >> 172.21.15.57:0/2097526259 conn(0x7f7f74095d30 msgr2=0x7f7f740b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 shutdown_connections 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.734+0000 7f7f79848700 1 -- 172.21.15.57:0/2097526259 wait complete. 2023-10-07T23:21:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:36.750 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:36.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.854+0000 7f53b9301700 1 Processor -- start 2023-10-07T23:21:36.861 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.854+0000 7f53b9301700 1 -- start start 2023-10-07T23:21:36.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b40cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53b40ca210 0x7f53b41598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40cb560 con 0x7f53b4159e30 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b40c9d80 con 0x7f53b40ca210 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40ca680 con 0x7f53b40cbb30 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53b40ca210 0x7f53b41598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b37fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f53b4159e30 0x7f53b4155730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:34134/0 (socket says 172.21.15.57:34134) 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53b40ca210 0x7f53b41598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34286/0 (socket says 172.21.15.57:34286) 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b37fe700 1 -- 172.21.15.57:0/2686771648 learned_addr learned my addr 172.21.15.57:0/2686771648 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b2ffd700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b40cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 -- 172.21.15.57:0/2686771648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 msgr2=0x7f53b40cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b40cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 -- 172.21.15.57:0/2686771648 >> v1:172.21.15.57:6789/0 conn(0x7f53b4159e30 legacy=0x7f53b4155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b1ffb700 1 -- 172.21.15.57:0/2686771648 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216368674 0 0) 0x7f53b40cb560 con 0x7f53b4159e30 2023-10-07T23:21:36.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 -- 172.21.15.57:0/2686771648 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b415a2b0 con 0x7f53b40ca210 2023-10-07T23:21:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b27fc700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b41598f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53a80098e0 tx=0x7f53a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd5d1cf0a2f4885 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b1ffb700 1 -- 172.21.15.57:0/2686771648 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800f6b0 con 0x7f53b40ca210 2023-10-07T23:21:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b1ffb700 1 -- 172.21.15.57:0/2686771648 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800f810 con 0x7f53b40ca210 2023-10-07T23:21:36.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b1ffb700 1 -- 172.21.15.57:0/2686771648 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800fbb0 con 0x7f53b40ca210 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- 172.21.15.57:0/2686771648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 msgr2=0x7f53b41598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b41598f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53a80098e0 tx=0x7f53a80091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- 172.21.15.57:0/2686771648 shutdown_connections 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b41598f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 --2- 172.21.15.57:0/2686771648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b40cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- 172.21.15.57:0/2686771648 >> 172.21.15.57:0/2686771648 conn(0x7f53b40670e0 msgr2=0x7f53b40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- 172.21.15.57:0/2686771648 shutdown_connections 2023-10-07T23:21:36.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.858+0000 7f53b9301700 1 -- 172.21.15.57:0/2686771648 wait complete. 2023-10-07T23:21:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 Processor -- start 2023-10-07T23:21:36.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- start start 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53b4159e30 0x7f53b4150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40c6070 con 0x7f53b40ca210 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41546e0 con 0x7f53b4159e30 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41549c0 con 0x7f53b40cbb30 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.868 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53b4159e30 0x7f53b4150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34300/0 (socket says 172.21.15.57:34300) 2023-10-07T23:21:36.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b27fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.869 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 -- 172.21.15.57:0/3188378509 learned_addr learned my addr 172.21.15.57:0/3188378509 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 -- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 msgr2=0x7f53b406b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b406b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 -- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53b4159e30 msgr2=0x7f53b4150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53b4159e30 0x7f53b4150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 -- 172.21.15.57:0/3188378509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8009420 con 0x7f53b40ca210 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b2ffd700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53a400ec80 tx=0x7f53a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400c850 con 0x7f53b40ca210 2023-10-07T23:21:36.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a400c9b0 con 0x7f53b40ca210 2023-10-07T23:21:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b4154ca0 con 0x7f53b40ca210 2023-10-07T23:21:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400cd50 con 0x7f53b40ca210 2023-10-07T23:21:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.862+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b40c3320 con 0x7f53b40ca210 2023-10-07T23:21:36.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.866+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b40711c0 con 0x7f53b40ca210 2023-10-07T23:21:36.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.866+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f53a4013680 con 0x7f53b40ca210 2023-10-07T23:21:36.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.866+0000 7f539affd700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 0x7f53a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:36.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.866+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53a407cd40 con 0x7f53b40ca210 2023-10-07T23:21:36.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.866+0000 7f53b27fc700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 0x7f53a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:36.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.870+0000 7f53b27fc700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 0x7f53a0062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53a80093f0 tx=0x7f53a8019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:36.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:36.874+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f53a404d330 con 0x7f53b40ca210 2023-10-07T23:21:37.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.034+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53b415a500 con 0x7f53b40ca210 2023-10-07T23:21:37.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f539affd700 1 -- 172.21.15.57:0/3188378509 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f53a4050950 con 0x7f53b40ca210 2023-10-07T23:21:37.042 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:37.042 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 msgr2=0x7f53a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 0x7f53a0062a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53a80093f0 tx=0x7f53a8019000 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 msgr2=0x7f53b40c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f53a400ec80 tx=0x7f53a40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 shutdown_connections 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53a0060550 0x7f53a0062a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53b4159e30 0x7f53b4150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53b40cbb30 0x7f53b406b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 --2- 172.21.15.57:0/3188378509 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53b40ca210 0x7f53b40c5b30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 >> 172.21.15.57:0/3188378509 conn(0x7f53b40670e0 msgr2=0x7f53b406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 shutdown_connections 2023-10-07T23:21:37.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.038+0000 7f53b9301700 1 -- 172.21.15.57:0/3188378509 wait complete. 2023-10-07T23:21:37.056 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 Processor -- start 2023-10-07T23:21:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 -- start start 2023-10-07T23:21:37.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f19c8157d90 0x7f19c80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c806ae80 con 0x7f19c8151090 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c815ae60 con 0x7f19c8157d90 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c8bd0 con 0x7f19c80c8750 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f19c8157d90 0x7f19c80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f19c8157d90 0x7f19c80c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34314/0 (socket says 172.21.15.57:34314) 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 -- 172.21.15.57:0/3619939187 learned_addr learned my addr 172.21.15.57:0/3619939187 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19c6d9d700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 -- 172.21.15.57:0/3619939187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 msgr2=0x7f19c806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 -- 172.21.15.57:0/3619939187 >> v1:172.21.15.154:6789/0 conn(0x7f19c8151090 legacy=0x7f19c815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 -- 172.21.15.57:0/3619939187 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c815b050 con 0x7f19c8157d90 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.154+0000 7f19bffff700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c80c8210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19b00099b0 tx=0x7f19b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0c7087972fe9e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c5d9b700 1 -- 172.21.15.57:0/3619939187 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0009ea0 con 0x7f19c8157d90 2023-10-07T23:21:37.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c5d9b700 1 -- 172.21.15.57:0/3619939187 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000b860 con 0x7f19c8157d90 2023-10-07T23:21:37.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c5d9b700 1 -- 172.21.15.57:0/3619939187 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000bc00 con 0x7f19c8157d90 2023-10-07T23:21:37.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 -- 172.21.15.57:0/3619939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 msgr2=0x7f19c80c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c80c8210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19b00099b0 tx=0x7f19b0009250 comp rx=0 tx=0).stop 2023-10-07T23:21:37.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 -- 172.21.15.57:0/3619939187 shutdown_connections 2023-10-07T23:21:37.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 --2- 172.21.15.57:0/3619939187 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c80c8210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 -- 172.21.15.57:0/3619939187 >> 172.21.15.57:0/3619939187 conn(0x7f19c80670e0 msgr2=0x7f19c80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 -- 172.21.15.57:0/3619939187 shutdown_connections 2023-10-07T23:21:37.164 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 -- 172.21.15.57:0/3619939187 wait complete. 2023-10-07T23:21:37.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.158+0000 7f19c759e700 1 Processor -- start 2023-10-07T23:21:37.165 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- start start 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c80afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19c8151090 0x7f19c8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c815aa50 con 0x7f19c8157d90 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c815a500 con 0x7f19c8151090 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c2e40 con 0x7f19c80c8750 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34324/0 (socket says 172.21.15.57:34324) 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 -- 172.21.15.57:0/1352649457 learned_addr learned my addr 172.21.15.57:0/1352649457 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c659c700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c80afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19bffff700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19c8151090 0x7f19c8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 -- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 msgr2=0x7f19c80afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c80afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 -- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19c8151090 msgr2=0x7f19c8156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19c8151090 0x7f19c8156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 -- 172.21.15.57:0/1352649457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00094f0 con 0x7f19c8157d90 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c6d9d700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ec0 tx=0x7f19c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000cb70 con 0x7f19c8157d90 2023-10-07T23:21:37.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c000f040 con 0x7f19c8157d90 2023-10-07T23:21:37.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c8159f40 con 0x7f19c8157d90 2023-10-07T23:21:37.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0014480 con 0x7f19c8157d90 2023-10-07T23:21:37.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.162+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c80c2ad0 con 0x7f19c8157d90 2023-10-07T23:21:37.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c80711c0 con 0x7f19c8157d90 2023-10-07T23:21:37.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f19c00147f0 con 0x7f19c8157d90 2023-10-07T23:21:37.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19be7fc700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 0x7f19a8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f19c007d280 con 0x7f19c8157d90 2023-10-07T23:21:37.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19c659c700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 0x7f19a8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.166+0000 7f19c659c700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 0x7f19a8062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19b80039f0 tx=0x7f19b8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.174+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19c004d870 con 0x7f19c8157d90 2023-10-07T23:21:37.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.366+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19c80c2f80 con 0x7f19c8157d90 2023-10-07T23:21:37.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.366+0000 7f19be7fc700 1 -- 172.21.15.57:0/1352649457 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f19c0050e90 con 0x7f19c8157d90 2023-10-07T23:21:37.373 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:37.374 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 msgr2=0x7f19a8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 0x7f19a8062a00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19b80039f0 tx=0x7f19b8007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 msgr2=0x7f19c8156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009ec0 tx=0x7f19c000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 shutdown_connections 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19c8157d90 0x7f19c8156770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f19a8060550 0x7f19a8062a00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19c8151090 0x7f19c8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 --2- 172.21.15.57:0/1352649457 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19c80c8750 0x7f19c80afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 >> 172.21.15.57:0/1352649457 conn(0x7f19c80670e0 msgr2=0x7f19c806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 shutdown_connections 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.370+0000 7f19c759e700 1 -- 172.21.15.57:0/1352649457 wait complete. 2023-10-07T23:21:37.377 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:37.790 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T23:21:25.308405+0000', 'modified': '2023-10-07T23:21:27.951913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6837/2260010718', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 2260010718}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 2260010718}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T23:21:37.790 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T23:21:37.790 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-07T23:21:37.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 Processor -- start 2023-10-07T23:21:37.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 -- start start 2023-10-07T23:21:37.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9118159230 0x7f91180aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180caba0 con 0x7f9118158e60 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911806b2d0 con 0x7f91180b04a0 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.902+0000 7f911fe01700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911806bb20 con 0x7f9118159230 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9118159230 0x7f91180aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9118159230 0x7f91180aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:41888/0 (socket says 172.21.15.57:41888) 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 -- 172.21.15.57:0/2184752016 learned_addr learned my addr 172.21.15.57:0/2184752016 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911db9d700 1 --1- 172.21.15.57:0/2184752016 >> v1:172.21.15.57:6789/0 conn(0x7f9118158e60 0x7f911806abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:34146/0 (socket says 172.21.15.57:34146) 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 -- 172.21.15.57:0/2184752016 >> v1:172.21.15.154:6789/0 conn(0x7f91180b04a0 legacy=0x7f911806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 -- 172.21.15.57:0/2184752016 >> v1:172.21.15.57:6789/0 conn(0x7f9118158e60 legacy=0x7f911806abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911cb9b700 1 -- 172.21.15.57:0/2184752016 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038682856 0 0) 0x7f91180caba0 con 0x7f9118158e60 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 -- 172.21.15.57:0/2184752016 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911806b560 con 0x7f9118159230 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911d39c700 1 --2- 172.21.15.57:0/2184752016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9118159230 0x7f91180aff60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91140099b0 tx=0x7f9114009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a642b3eeb15dc62c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911cb9b700 1 -- 172.21.15.57:0/2184752016 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400b860 con 0x7f9118159230 2023-10-07T23:21:37.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911cb9b700 1 -- 172.21.15.57:0/2184752016 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400b9c0 con 0x7f9118159230 2023-10-07T23:21:37.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911cb9b700 1 -- 172.21.15.57:0/2184752016 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400bd60 con 0x7f9118159230 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 -- 172.21.15.57:0/2184752016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9118159230 msgr2=0x7f91180aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 --2- 172.21.15.57:0/2184752016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9118159230 0x7f91180aff60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91140099b0 tx=0x7f9114009250 comp rx=0 tx=0).stop 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 -- 172.21.15.57:0/2184752016 shutdown_connections 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 --2- 172.21.15.57:0/2184752016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9118159230 0x7f91180aff60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 -- 172.21.15.57:0/2184752016 >> 172.21.15.57:0/2184752016 conn(0x7f9118068740 msgr2=0x7f9118068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 -- 172.21.15.57:0/2184752016 shutdown_connections 2023-10-07T23:21:37.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.906+0000 7f911fe01700 1 -- 172.21.15.57:0/2184752016 wait complete. 2023-10-07T23:21:37.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.910+0000 7f911fe01700 1 Processor -- start 2023-10-07T23:21:37.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.910+0000 7f911fe01700 1 -- start start 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91180b04a0 0x7f9118158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 0x7f911816a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180a9340 con 0x7f9118158e60 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180a9620 con 0x7f91180b04a0 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180a9900 con 0x7f9118159230 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34334/0 (socket says 172.21.15.57:34334) 2023-10-07T23:21:37.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 -- 172.21.15.57:0/3648816144 learned_addr learned my addr 172.21.15.57:0/3648816144 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911db9d700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91180b04a0 0x7f9118158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911e39e700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 0x7f911816a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 -- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 msgr2=0x7f911816a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 0x7f911816a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 -- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91180b04a0 msgr2=0x7f9118158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91180b04a0 0x7f9118158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 -- 172.21.15.57:0/3648816144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140094f0 con 0x7f9118158e60 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911e39e700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 0x7f911816a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:37.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911d39c700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9114006f50 tx=0x7f9114017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f9109ffb700 1 -- 172.21.15.57:0/3648816144 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911401f070 con 0x7f9118158e60 2023-10-07T23:21:37.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f9109ffb700 1 -- 172.21.15.57:0/3648816144 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114011c60 con 0x7f9118158e60 2023-10-07T23:21:37.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91181696c0 con 0x7f9118158e60 2023-10-07T23:21:37.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f9109ffb700 1 -- 172.21.15.57:0/3648816144 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114025420 con 0x7f9118158e60 2023-10-07T23:21:37.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.914+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181587e0 con 0x7f9118158e60 2023-10-07T23:21:37.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f9109ffb700 1 -- 172.21.15.57:0/3648816144 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9114023070 con 0x7f9118158e60 2023-10-07T23:21:37.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f9109ffb700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 0x7f90fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f9109ffb700 1 -- 172.21.15.57:0/3648816144 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f911407d8b0 con 0x7f9118158e60 2023-10-07T23:21:37.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f911db9d700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 0x7f90fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f911db9d700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 0x7f90fc062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f910c007fd0 tx=0x7f910c007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f91220aa740 1 Processor -- start 2023-10-07T23:21:37.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.918+0000 7f91220aa740 1 -- start start 2023-10-07T23:21:37.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1c0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1c0e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b3bf30 con 0x1990710 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b37ed0 con 0x1990ae0 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1afe0b0 con 0x19db790 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34346/0 (socket says 172.21.15.57:34346) 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 -- 172.21.15.57:0/2045746316 learned_addr learned my addr 172.21.15.57:0/2045746316 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911db9d700 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1c0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911e39e700 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1c0e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 -- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 msgr2=0x1c0e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1c0e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 -- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 msgr2=0x1c0df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1c0df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 -- 172.21.15.57:0/2045746316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x19dc9e0 con 0x1990710 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f911d39c700 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9114009fd0 tx=0x7f911407bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99ecedcebc6b511 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f9102ffd700 1 -- 172.21.15.57:0/2045746316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114054df0 con 0x1990710 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f9102ffd700 1 -- 172.21.15.57:0/2045746316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114058b40 con 0x1990710 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f9102ffd700 1 -- 172.21.15.57:0/2045746316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911404fd50 con 0x1990710 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- 172.21.15.57:0/2045746316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 msgr2=0x1c0e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9114009fd0 tx=0x7f911407bd30 comp rx=0 tx=0).stop 2023-10-07T23:21:37.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- 172.21.15.57:0/2045746316 shutdown_connections 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1c0e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1c0e440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 --2- 172.21.15.57:0/2045746316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1c0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- 172.21.15.57:0/2045746316 >> 172.21.15.57:0/2045746316 conn(0x1990eb0 msgr2=0x1bca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- 172.21.15.57:0/2045746316 shutdown_connections 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.922+0000 7f91220aa740 1 -- 172.21.15.57:0/2045746316 wait complete. 2023-10-07T23:21:37.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 -- start start 2023-10-07T23:21:37.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 Processor -- start 2023-10-07T23:21:37.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c12510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x1afdd50 con 0x1990ae0 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ad6970 con 0x1990710 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b87480 con 0x19db790 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f91220aa740 10 client.0 _ll_register_callbacks cb 0x1b816a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34362/0 (socket says 172.21.15.57:34362) 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 -- 172.21.15.57:0/2641263156 learned_addr learned my addr 172.21.15.57:0/2641263156 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911db9d700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c12510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911e39e700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 -- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 msgr2=0x1bf3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 -- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 msgr2=0x1c12510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c12510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 -- 172.21.15.57:0/2641263156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91181587e0 con 0x1990ae0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f911d39c700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f911407beb0 tx=0x7f9114064000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e917d27493fb8f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.926+0000 7f90f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114051100 con 0x1990ae0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9114051260 con 0x1990ae0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ad6cc0 con 0x1990ae0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91140588c0 con 0x1990ae0 2023-10-07T23:21:37.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 10 client.4659 fetch_fsmap learned FSMap version 6 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x1b06370 con 0x1990ae0 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ad6cc0 con 0x1990ae0 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9114052bb0 con 0x1990ae0 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 10 client.4659 fetch_fsmap finished waiting for FSMap version 6 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 20 client.4659 populate_metadata read hostname 'smithi057' 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bf53b0 0x1c1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 4 client.4659 mds_command: new command op to 4294 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 --> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1ada3e0 con 0x1bf53b0 2023-10-07T23:21:37.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f911db9d700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bf53b0 0x1c1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f911db9d700 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bf53b0 0x1c1d6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6d9c248e97d993 server_cookie=575b0ebb58dd94e7 in_seq=0 out_seq=0 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 10 client.4659 ms_handle_connect on v2:172.21.15.154:6836/2260010718 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 1 -- 172.21.15.57:0/2641263156 <== mds.0 v2:172.21.15.154:6836/2260010718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x1ada3e0 con 0x1bf53b0 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 10 client.4659 handle_command_reply: tid=0 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 client.4659 shutdown 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 msgr2=0x1c12a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f911407beb0 tx=0x7f9114064000 comp rx=0 tx=0).stop 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 shutdown_connections 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bf53b0 0x1c1d6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c12a50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 --2- 172.21.15.57:0/2641263156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c12510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f90f7fff700 0 client.4659 ms_handle_reset on v2:172.21.15.154:6836/2260010718 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 >> 172.21.15.57:0/2641263156 conn(0x1990eb0 msgr2=0x1bc2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 shutdown_connections 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.930+0000 7f91220aa740 1 -- 172.21.15.57:0/2641263156 wait complete. 2023-10-07T23:21:37.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.934+0000 7f91220aa740 20 client.4659 trim_cache size 0 max 16384 2023-10-07T23:21:37.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 Processor -- start 2023-10-07T23:21:37.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- start start 2023-10-07T23:21:37.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1bc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ae6c90 con 0x1990710 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b3b9d0 con 0x1990ae0 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1b3cef0 con 0x19db790 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34370/0 (socket says 172.21.15.57:34370) 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 -- 172.21.15.57:0/886429280 learned_addr learned my addr 172.21.15.57:0/886429280 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911d39c700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1bc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911e39e700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 -- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 msgr2=0x1bf1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 -- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 msgr2=0x1bc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1bc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 -- 172.21.15.57:0/886429280 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1aecd80 con 0x1990710 2023-10-07T23:21:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911e39e700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:37.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f911db9d700 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f910c0019f0 tx=0x7f910c028250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c745ce0656524d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91037fe700 1 -- 172.21.15.57:0/886429280 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0133f0 con 0x1990710 2023-10-07T23:21:37.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91037fe700 1 -- 172.21.15.57:0/886429280 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c026070 con 0x1990710 2023-10-07T23:21:37.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91037fe700 1 -- 172.21.15.57:0/886429280 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c013550 con 0x1990710 2023-10-07T23:21:37.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- 172.21.15.57:0/886429280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 msgr2=0x1bc4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f910c0019f0 tx=0x7f910c028250 comp rx=0 tx=0).stop 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- 172.21.15.57:0/886429280 shutdown_connections 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bf1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990ae0 0x1bc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 --2- 172.21.15.57:0/886429280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990710 0x1bc4ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- 172.21.15.57:0/886429280 >> 172.21.15.57:0/886429280 conn(0x1990eb0 msgr2=0x1c0d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- 172.21.15.57:0/886429280 shutdown_connections 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.954+0000 7f91220aa740 1 -- 172.21.15.57:0/886429280 wait complete. 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 -- start start 2023-10-07T23:21:37.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 Processor -- start 2023-10-07T23:21:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bc64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b0bf40 con 0x1990ae0 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b8e6d0 con 0x1990710 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a74430 con 0x19db790 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f91220aa740 10 client.0 _ll_register_callbacks cb 0x195c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34380/0 (socket says 172.21.15.57:34380) 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 -- 172.21.15.57:0/2443298270 learned_addr learned my addr 172.21.15.57:0/2443298270 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911db9d700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911e39e700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bc64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 -- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 msgr2=0x1bc64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bc64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 -- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 msgr2=0x1c676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 -- 172.21.15.57:0/2443298270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c001860 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911e39e700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bc64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f911d39c700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9114009fd0 tx=0x7f9114066070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35da8b286698465 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.958+0000 7f90f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114051570 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9114052bb0 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9114058b40 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9114051310 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 10 client.4665 fetch_fsmap learned FSMap version 6 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x1ae1130 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114058b40 con 0x1990ae0 2023-10-07T23:21:37.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f91140760a0 con 0x1990ae0 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 10 client.4665 fetch_fsmap finished waiting for FSMap version 6 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 10 client.4665 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 20 client.4665 populate_metadata read hostname 'smithi057' 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bc6e80 0x1bc7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 4 client.4665 mds_command: new command op to 4294 tid=0[{"prefix": "status"}] 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 --> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1b0c410 con 0x1bc6e80 2023-10-07T23:21:37.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f911db9d700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bc6e80 0x1bc7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:37.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f911db9d700 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bc6e80 0x1bc7250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c164732c504558 server_cookie=558ef5d2e31f665a in_seq=0 out_seq=0 2023-10-07T23:21:37.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 10 client.4665 ms_handle_connect on v2:172.21.15.154:6836/2260010718 2023-10-07T23:21:37.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 1 -- 172.21.15.57:0/2443298270 <== mds.0 v2:172.21.15.154:6836/2260010718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x1b0c410 con 0x1bc6e80 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 10 client.4665 handle_command_reply: tid=0 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 client.4665 shutdown 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 msgr2=0x1c67c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9114009fd0 tx=0x7f9114066070 comp rx=0 tx=0).stop 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 shutdown_connections 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:6836/2260010718,v1:172.21.15.154:6837/2260010718] conn(0x1bc6e80 0x1bc7250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x19db790 0x1bc64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f90f7fff700 0 client.4665 ms_handle_reset on v2:172.21.15.154:6836/2260010718 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x1990ae0 0x1c67c20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 --2- 172.21.15.57:0/2443298270 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x1990710 0x1c676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 >> 172.21.15.57:0/2443298270 conn(0x1990eb0 msgr2=0x1c0e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 shutdown_connections 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 1 -- 172.21.15.57:0/2443298270 wait complete. 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.962+0000 7f91220aa740 20 client.4665 trim_cache size 0 max 16384 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "aaf09e93-912b-470e-9df7-bb9a606f50e4", 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4294, 2023-10-07T23:21:37.972 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 12.029078834, 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 6, 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 23, 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 23, 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 19.484760736999998 2023-10-07T23:21:37.973 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 msgr2=0x7f90fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 0x7f90fc062a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f910c007fd0 tx=0x7f910c007d50 comp rx=0 tx=0).stop 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 msgr2=0x7f9118169be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9114006f50 tx=0x7f9114017000 comp rx=0 tx=0).stop 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 shutdown_connections 2023-10-07T23:21:37.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9118159230 0x7f911816a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9118158e60 0x7f9118169be0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90fc060550 0x7f90fc062a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 --2- 172.21.15.57:0/3648816144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91180b04a0 0x7f9118158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 >> 172.21.15.57:0/3648816144 conn(0x7f9118068740 msgr2=0x7f91180b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 shutdown_connections 2023-10-07T23:21:37.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:37.966+0000 7f911fe01700 1 -- 172.21.15.57:0/3648816144 wait complete. 2023-10-07T23:21:37.985 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-07T23:21:38.006 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-07T23:21:38.006 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-07T23:21:38.007 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-07T23:21:38.007 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-07T23:21:38.007 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-07T23:21:38.007 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-07T23:21:38.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.007 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:21:38.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.015 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-07T23:21:38.035 INFO:teuthology.orchestra.run.smithi057.stderr:Cannot find device "ceph-brx" 2023-10-07T23:21:38.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:21:38.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.037 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:21:38.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-07T23:21:38.059 INFO:teuthology.orchestra.run.smithi154.stderr:Cannot find device "ceph-brx" 2023-10-07T23:21:38.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:21:38.060 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-07T23:21:38.061 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:21:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 Processor -- start 2023-10-07T23:21:38.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 -- start start 2023-10-07T23:21:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9f480cb6d0 0x7f9f48155b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f4815a210 0x7f9f480c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480ca6c0 con 0x7f9f48155780 2023-10-07T23:21:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480cb240 con 0x7f9f480cb6d0 2023-10-07T23:21:38.181 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4df6c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480ca910 con 0x7f9f4815a210 2023-10-07T23:21:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f3ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9f480cb6d0 0x7f9f48155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f3ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9f480cb6d0 0x7f9f48155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34394/0 (socket says 172.21.15.57:34394) 2023-10-07T23:21:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f3ffff700 1 -- 172.21.15.57:0/840444316 learned_addr learned my addr 172.21.15.57:0/840444316 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:38.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.174+0000 7f9f4d76b700 1 --2- 172.21.15.57:0/840444316 >> v2:172.21.15.154:3300/0 conn(0x7f9f4815a210 0x7f9f480c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3ffff700 1 -- 172.21.15.57:0/840444316 >> v2:172.21.15.154:3300/0 conn(0x7f9f4815a210 msgr2=0x7f9f480c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3ffff700 1 --2- 172.21.15.57:0/840444316 >> v2:172.21.15.154:3300/0 conn(0x7f9f4815a210 0x7f9f480c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3ffff700 1 -- 172.21.15.57:0/840444316 >> v1:172.21.15.154:6789/0 conn(0x7f9f48155780 legacy=0x7f9f48069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:38.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3ffff700 1 -- 172.21.15.57:0/840444316 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480cabf0 con 0x7f9f480cb6d0 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3ffff700 1 --2- 172.21.15.57:0/840444316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f480cb6d0 0x7f9f48155b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f480c7f00 tx=0x7f9f38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97223ee2b616bd30 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3f7fe700 1 -- 172.21.15.57:0/840444316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800b820 con 0x7f9f480cb6d0 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3f7fe700 1 -- 172.21.15.57:0/840444316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3800b980 con 0x7f9f480cb6d0 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f3f7fe700 1 -- 172.21.15.57:0/840444316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800bd20 con 0x7f9f480cb6d0 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- 172.21.15.57:0/840444316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f480cb6d0 msgr2=0x7f9f48155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/840444316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f480cb6d0 0x7f9f48155b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f480c7f00 tx=0x7f9f38009250 comp rx=0 tx=0).stop 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- 172.21.15.57:0/840444316 shutdown_connections 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/840444316 >> v2:172.21.15.154:3300/0 conn(0x7f9f4815a210 0x7f9f480c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/840444316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f480cb6d0 0x7f9f48155b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- 172.21.15.57:0/840444316 >> 172.21.15.57:0/840444316 conn(0x7f9f4806b5d0 msgr2=0x7f9f4806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- 172.21.15.57:0/840444316 shutdown_connections 2023-10-07T23:21:38.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- 172.21.15.57:0/840444316 wait complete. 2023-10-07T23:21:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 Processor -- start 2023-10-07T23:21:38.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.178+0000 7f9f4df6c700 1 -- start start 2023-10-07T23:21:38.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 0x7f9f480c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f4815a210 0x7f9f480b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480af890 con 0x7f9f48155780 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480c6420 con 0x7f9f4815a210 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480afaa0 con 0x7f9f480cb6d0 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34396/0 (socket says 172.21.15.57:34396) 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 -- 172.21.15.57:0/3435307708 learned_addr learned my addr 172.21.15.57:0/3435307708 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4d76b700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f4815a210 0x7f9f480b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4cf6a700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 0x7f9f480c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 -- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 msgr2=0x7f9f480c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 0x7f9f480c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 -- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f4815a210 msgr2=0x7f9f480b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f4815a210 0x7f9f480b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 -- 172.21.15.57:0/3435307708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f380094f0 con 0x7f9f48155780 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4cf6a700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 0x7f9f480c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:38.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3ffff700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f380030c0 tx=0x7f9f38016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38011db0 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38020070 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480afd80 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3801b6a0 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f480b0340 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.182+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f480711c0 con 0x7f9f48155780 2023-10-07T23:21:38.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.186+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9f38022030 con 0x7f9f48155780 2023-10-07T23:21:38.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.186+0000 7f9f3cff9700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 0x7f9f34062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.186+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f3807e680 con 0x7f9f48155780 2023-10-07T23:21:38.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.186+0000 7f9f4cf6a700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 0x7f9f34062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:38.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.186+0000 7f9f4cf6a700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 0x7f9f34062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f44002930 tx=0x7f9f44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.194+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9f3804ec70 con 0x7f9f48155780 2023-10-07T23:21:38.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.390+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f480711c0 con 0x7f9f48155780 2023-10-07T23:21:38.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.390+0000 7f9f3cff9700 1 -- 172.21.15.57:0/3435307708 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9f38052290 con 0x7f9f48155780 2023-10-07T23:21:38.396 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:21:38.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 msgr2=0x7f9f34062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:38.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 0x7f9f34062a00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f44002930 tx=0x7f9f44009040 comp rx=0 tx=0).stop 2023-10-07T23:21:38.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 msgr2=0x7f9f48150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f380030c0 tx=0x7f9f38016000 comp rx=0 tx=0).stop 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 shutdown_connections 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9f34060550 0x7f9f34062a00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f4815a210 0x7f9f480b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f48155780 0x7f9f48150490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 --2- 172.21.15.57:0/3435307708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9f480cb6d0 0x7f9f480c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 >> 172.21.15.57:0/3435307708 conn(0x7f9f4806b5d0 msgr2=0x7f9f480cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 shutdown_connections 2023-10-07T23:21:38.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:38.394+0000 7f9f4df6c700 1 -- 172.21.15.57:0/3435307708 wait complete. 2023-10-07T23:21:38.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:21:38.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:21:38.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-07T23:21:38.410 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T23:21:38.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:38.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:21:38.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:21:38.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:21:38.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:21:38.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.411 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:38.419 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42537sec preferred_lft 42537sec 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 02:42:22:80:74:34 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:38.420 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:21:38.421 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:38.421 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T23:21:38.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.421 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:38.421 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:38.421 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2023-10-07T23:21:38.421 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2023-10-07T23:21:38.421 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2023-10-07T23:21:38.422 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T23:21:38.422 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:38.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.503 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:21:38.512 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-07T23:21:38.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.513 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-07T23:21:38.533 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:21:38.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T23:21:38.534 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:38.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.624 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:21:38.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.629 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-10-07T23:21:38.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.633 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:38.633 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:38.633 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:21:38.633 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-07T23:21:38.633 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:38.659 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-07T23:21:38.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-07T23:21:38.659 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-07T23:21:38.660 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:38.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:38.771 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:38.771 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:38.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-07T23:21:38.771 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T23:21:38.772 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:38.796 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T23:21:38.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:38.797 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:38.804 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:21:38.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:38.804 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:38.811 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:38.871 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:38.871 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:38.885 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:38.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:38.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:38.887 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:38.894 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:21:38.894 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T23:21:38.897 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:38.915 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:38.915 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:38.971 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:38.972 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:38.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:38.973 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:38.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:21:38.978+0000 7fae86a8a380 -1 init, newargv = 0x558258ab3f40 newargc=15 2023-10-07T23:21:38.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[39978]: starting ceph client 2023-10-07T23:21:38.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[39978]: starting fuse 2023-10-07T23:21:40.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:40.097 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:40.098 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:40.157 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:40.158 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:40.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:40.159 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:40.165 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:21:40.166 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:21:40.166 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:21:40.167 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:21:40.168 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:21:40.168 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:21:40.168 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:40.168 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-10-07T23:21:40.168 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:21:40.250 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.39978.asok 2023-10-07T23:21:40.436 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39978.asok 2023-10-07T23:21:40.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:40.436 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39978.asok status 2023-10-07T23:21:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "39978", 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-07T23:21:40.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4680, 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 4680 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:21:40.535 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1933158952 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1933158952 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.4680 192.168.0.1:0/1933158952", 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/1933158952", 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 6, 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 23, 2023-10-07T23:21:40.536 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:21:40.537 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-07T23:21:40.537 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-07T23:21:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:21:40.543 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:21:40.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 Processor -- start 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 -- start start 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faac0158fb0 0x7faac00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac006b3c0 con 0x7faac006b850 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac006a7b0 con 0x7faac0158fb0 2023-10-07T23:21:40.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabf59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac006aa90 con 0x7faac0158be0 2023-10-07T23:21:40.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faac0158fb0 0x7faac00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7faac0158fb0 0x7faac00ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34410/0 (socket says 172.21.15.57:34410) 2023-10-07T23:21:40.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 -- 172.21.15.57:0/30013765 learned_addr learned my addr 172.21.15.57:0/30013765 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:40.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabe59c700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 -- 172.21.15.57:0/30013765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 msgr2=0x7faac00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:40.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 -- 172.21.15.57:0/30013765 >> v1:172.21.15.154:6789/0 conn(0x7faac006b850 legacy=0x7faac00cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.650+0000 7faabdd9b700 1 -- 172.21.15.57:0/30013765 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac006ad70 con 0x7faac0158fb0 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabdd9b700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac00ce6b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faaa80099b0 tx=0x7faaa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936715e65c03aadf server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabd59a700 1 -- 172.21.15.57:0/30013765 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8009ea0 con 0x7faac0158fb0 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabd59a700 1 -- 172.21.15.57:0/30013765 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa800b860 con 0x7faac0158fb0 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabd59a700 1 -- 172.21.15.57:0/30013765 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800bc00 con 0x7faac0158fb0 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- 172.21.15.57:0/30013765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 msgr2=0x7faac00ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac00ce6b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faaa80099b0 tx=0x7faaa8009250 comp rx=0 tx=0).stop 2023-10-07T23:21:40.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- 172.21.15.57:0/30013765 shutdown_connections 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac00ce6b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 --2- 172.21.15.57:0/30013765 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- 172.21.15.57:0/30013765 >> 172.21.15.57:0/30013765 conn(0x7faac0068520 msgr2=0x7faac0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- 172.21.15.57:0/30013765 shutdown_connections 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- 172.21.15.57:0/30013765 wait complete. 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 Processor -- start 2023-10-07T23:21:40.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.654+0000 7faabf59e700 1 -- start start 2023-10-07T23:21:40.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faac006b850 0x7faac015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b35f0 con 0x7faac0158fb0 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b3870 con 0x7faac006b850 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b3b50 con 0x7faac0158be0 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34414/0 (socket says 172.21.15.57:34414) 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 -- 172.21.15.57:0/1755580105 learned_addr learned my addr 172.21.15.57:0/1755580105 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabdd9b700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabe59c700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faac006b850 0x7faac015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 -- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 msgr2=0x7faac00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 -- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faac006b850 msgr2=0x7faac015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faac006b850 0x7faac015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 -- 172.21.15.57:0/1755580105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80094f0 con 0x7faac0158fb0 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabdd9b700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:40.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabed9d700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faab800c4c0 tx=0x7faab8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8011590 con 0x7faac0158fb0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8011710 con 0x7faac0158fb0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac00b3e30 con 0x7faac0158fb0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8011ad0 con 0x7faac0158fb0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac00b43f0 con 0x7faac0158fb0 2023-10-07T23:21:40.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.658+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac00b45e0 con 0x7faac0158fb0 2023-10-07T23:21:40.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.662+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7faab801b3b0 con 0x7faac0158fb0 2023-10-07T23:21:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.662+0000 7faab67fc700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 0x7faaa4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.662+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faab807bc50 con 0x7faac0158fb0 2023-10-07T23:21:40.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.662+0000 7faabe59c700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 0x7faaa4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:40.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.662+0000 7faabe59c700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 0x7faaa4062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faac00cd350 tx=0x7faab000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:40.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.670+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faab8081020 con 0x7faac0158fb0 2023-10-07T23:21:40.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.866+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faac00711c0 con 0x7faac0158fb0 2023-10-07T23:21:40.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.866+0000 7faab67fc700 1 -- 172.21.15.57:0/1755580105 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7faab8017020 con 0x7faac0158fb0 2023-10-07T23:21:40.873 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 msgr2=0x7faaa4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 0x7faaa4062a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faac00cd350 tx=0x7faab000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 msgr2=0x7faac01690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faab800c4c0 tx=0x7faab8007f50 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 shutdown_connections 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faac0158fb0 0x7faac01690c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7faac0158be0 0x7faac00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7faaa4060550 0x7faaa4062a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 --2- 172.21.15.57:0/1755580105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faac006b850 0x7faac015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:40.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 >> 172.21.15.57:0/1755580105 conn(0x7faac0068520 msgr2=0x7faac01579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:40.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 shutdown_connections 2023-10-07T23:21:40.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:40.870+0000 7faabf59e700 1 -- 172.21.15.57:0/1755580105 wait complete. 2023-10-07T23:21:40.887 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:21:40.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:21:40.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:21:40.888 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-07T23:21:40.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.888 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.894 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42534sec preferred_lft 42534sec 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.895 INFO:teuthology.orchestra.run.smithi057.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 02:42:22:80:74:34 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 5a:76:86:e5:a8:05 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::88e7:63ff:fe49:2a21/64 scope link 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 5a:76:86:e5:a8:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:21:40.896 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::5876:86ff:fee5:a805/64 scope link 2023-10-07T23:21:40.897 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:40.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.897 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:21:40.908 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-07T23:21:40.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.910 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-07T23:21:40.919 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:21:40.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T23:21:40.920 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:40.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.959 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:21:40.964 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:21:40.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:40.965 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2023-10-07T23:21:41.012 INFO:teuthology.orchestra.run.smithi057.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-07T23:21:41.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:41.012 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:41.013 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:41.013 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T23:21:41.013 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-07T23:21:41.013 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:41.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:41.045 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-07T23:21:41.054 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:41.054 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether fa:8d:08:7b:63:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f88d:8ff:fe7b:63ba/64 scope link 2023-10-07T23:21:41.055 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:41.056 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-07T23:21:41.114 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:41.114 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:41.114 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether fa:8d:08:7b:63:ba brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::f88d:8ff:fe7b:63ba/64 scope link 2023-10-07T23:21:41.115 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:41.116 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-07T23:21:41.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-07T23:21:41.117 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:41.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:41.178 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:21:41.178 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:21:41.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.1 up 2023-10-07T23:21:41.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.1 master ceph-brx 2023-10-07T23:21:41.178 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:21:41.208 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-07T23:21:41.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:41.208 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:41.256 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-07T23:21:41.256 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:41.257 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:41.263 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:41.275 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:41.276 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:41.332 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:41.334 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:41.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:41.334 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:41.341 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:21:41.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-07T23:21:41.342 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-07T23:21:41.344 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:41.359 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:41.360 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:41.415 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:41.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:41.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:41.417 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:41.421 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:21:41.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:2023-10-07T23:21:41.422+0000 7f4d51ff1380 -1 init, newargv = 0x55abe4cacd00 newargc=15 2023-10-07T23:21:41.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:ceph-fuse[40120]: starting ceph client 2023-10-07T23:21:41.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi057.stderr:ceph-fuse[40120]: starting fuse 2023-10-07T23:21:42.524 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:21:42.541 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:42.541 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:42.599 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:42.600 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:42.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:42.601 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:21:42.607 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:21:42.608 INFO:teuthology.orchestra.run.smithi057.stdout:53 2023-10-07T23:21:42.608 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-07T23:21:42.608 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:21:42.608 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:21:42.608 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:21:42.608 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:21:42.609 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:21:42.610 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:21:42.611 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.2")) 2023-10-07T23:21:42.611 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:21:42.693 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.2.40120.asok 2023-10-07T23:21:42.875 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.40120.asok 2023-10-07T23:21:42.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:42.875 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.40120.asok status 2023-10-07T23:21:43.017 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:21:43.017 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-07T23:21:43.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:21:43.017 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "2", 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "40120", 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4692, 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:21:43.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 4692 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.2:0", 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1445132517 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.2:0", 2023-10-07T23:21:43.019 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1445132517 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.4692 192.168.0.2:0/1445132517", 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.2:0/1445132517", 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 6, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 23, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-07T23:21:43.020 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:21:43.027 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:21:43.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 Processor -- start 2023-10-07T23:21:43.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 -- start start 2023-10-07T23:21:43.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7f8062ff0 0x7fc7f81583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f806d750 con 0x7fc7f8062ba0 2023-10-07T23:21:43.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80c8b80 con 0x7fc7f8062ff0 2023-10-07T23:21:43.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fe514700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8068c40 con 0x7fc7f80c7800 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7fcab1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7f8062ff0 0x7fc7f81583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f77fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc7f8062ba0 0x7fc7f8068530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:55276/0 (socket says 172.21.15.57:55276) 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f77fe700 1 -- 172.21.15.57:0/618858930 learned_addr learned my addr 172.21.15.57:0/618858930 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f6ffd700 1 -- 172.21.15.57:0/618858930 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469233201 0 0) 0x7fc7e8001490 con 0x7fc7f8062ba0 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f6ffd700 1 -- 172.21.15.57:0/618858930 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003570 con 0x7fc7f8062ba0 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 -- 172.21.15.57:0/618858930 >> v2:172.21.15.154:3300/0 conn(0x7fc7f8062ff0 msgr2=0x7fc7f81583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 --2- 172.21.15.57:0/618858930 >> v2:172.21.15.154:3300/0 conn(0x7fc7f8062ff0 0x7fc7f81583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 -- 172.21.15.57:0/618858930 >> v1:172.21.15.154:6789/0 conn(0x7fc7f8062ba0 legacy=0x7fc7f8068530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 -- 172.21.15.57:0/618858930 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80c7f70 con 0x7fc7f80c7800 2023-10-07T23:21:43.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f7fff700 1 --2- 172.21.15.57:0/618858930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f806d210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00098e0 tx=0x7fc7e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b419fdcf5c6aa7e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:43.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f6ffd700 1 -- 172.21.15.57:0/618858930 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0011630 con 0x7fc7f80c7800 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f6ffd700 1 -- 172.21.15.57:0/618858930 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0011790 con 0x7fc7f80c7800 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.134+0000 7fc7f6ffd700 1 -- 172.21.15.57:0/618858930 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0011b30 con 0x7fc7f80c7800 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- 172.21.15.57:0/618858930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 msgr2=0x7fc7f806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- 172.21.15.57:0/618858930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f806d210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00098e0 tx=0x7fc7e00091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- 172.21.15.57:0/618858930 shutdown_connections 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- 172.21.15.57:0/618858930 >> v2:172.21.15.154:3300/0 conn(0x7fc7f8062ff0 0x7fc7f81583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- 172.21.15.57:0/618858930 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f806d210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- 172.21.15.57:0/618858930 >> 172.21.15.57:0/618858930 conn(0x7fc7f80cce30 msgr2=0x7fc7f80c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- 172.21.15.57:0/618858930 shutdown_connections 2023-10-07T23:21:43.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- 172.21.15.57:0/618858930 wait complete. 2023-10-07T23:21:43.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 Processor -- start 2023-10-07T23:21:43.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- start start 2023-10-07T23:21:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f8062ff0 0x7fc7f814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f8167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80c3410 con 0x7fc7f8062ba0 2023-10-07T23:21:43.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80c5cb0 con 0x7fc7f8062ff0 2023-10-07T23:21:43.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.138+0000 7fc7fe514700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80c5f90 con 0x7fc7f80c7800 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f8062ff0 0x7fc7f814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34426/0 (socket says 172.21.15.57:34426) 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 -- 172.21.15.57:0/335605822 learned_addr learned my addr 172.21.15.57:0/335605822 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7fcab1700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f8167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 -- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 msgr2=0x7fc7f8167980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f8167980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 -- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f8062ff0 msgr2=0x7fc7f814a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f8062ff0 0x7fc7f814a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 -- 172.21.15.57:0/335605822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0009420 con 0x7fc7f8062ba0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7f7fff700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8068d80 tx=0x7fc7e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0000c40 con 0x7fc7f8062ba0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0011900 con 0x7fc7f8062ba0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0025420 con 0x7fc7f8062ba0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f80c6270 con 0x7fc7f8062ba0 2023-10-07T23:21:43.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f80c5940 con 0x7fc7f8062ba0 2023-10-07T23:21:43.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f80673f0 con 0x7fc7f8062ba0 2023-10-07T23:21:43.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.142+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc7e0023070 con 0x7fc7f8062ba0 2023-10-07T23:21:43.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.146+0000 7fc7effff700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 0x7fc7d8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:43.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.146+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc7e007dcb0 con 0x7fc7f8062ba0 2023-10-07T23:21:43.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.146+0000 7fc7f77fe700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 0x7fc7d8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:43.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.146+0000 7fc7f77fe700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 0x7fc7d8062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc7f814d1e0 tx=0x7fc7e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:43.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.150+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc7f80673f0 con 0x7fc7f8062ba0 2023-10-07T23:21:43.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc7f80673f0 con 0x7fc7f8062ba0 2023-10-07T23:21:43.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7effff700 1 -- 172.21.15.57:0/335605822 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc7e002a060 con 0x7fc7f8062ba0 2023-10-07T23:21:43.351 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 msgr2=0x7fc7d8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 0x7fc7d8062a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc7f814d1e0 tx=0x7fc7e8009040 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 msgr2=0x7fc7f80c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.346+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8068d80 tx=0x7fc7e0017000 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 shutdown_connections 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc7f80c7800 0x7fc7f8167980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc7d8060550 0x7fc7d8062a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7f8062ff0 0x7fc7f814a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 --2- 172.21.15.57:0/335605822 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc7f8062ba0 0x7fc7f80c5400 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:43.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 >> 172.21.15.57:0/335605822 conn(0x7fc7f80cce30 msgr2=0x7fc7f80ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:43.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 shutdown_connections 2023-10-07T23:21:43.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:43.350+0000 7fc7fe514700 1 -- 172.21.15.57:0/335605822 wait complete. 2023-10-07T23:21:43.365 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:21:43.365 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:21:43.366 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:21:43.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.366 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-07T23:21:43.374 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:43.375 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42518sec preferred_lft 42518sec 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:9b:82:8d:1f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:21:43.376 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:43.376 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-07T23:21:43.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-07T23:21:43.377 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:43.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.425 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:21:43.437 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-07T23:21:43.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.439 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-07T23:21:43.465 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:21:43.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:21:43.467 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:43.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.551 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:21:43.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.558 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-10-07T23:21:43.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.565 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:43.565 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:43.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T23:21:43.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-07T23:21:43.566 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:43.593 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-07T23:21:43.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.593 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:43.593 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:43.593 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-07T23:21:43.593 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-07T23:21:43.593 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-07T23:21:43.594 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-07T23:21:43.594 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-07T23:21:43.594 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:43.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:43.680 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:43.680 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:43.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-10-07T23:21:43.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2023-10-07T23:21:43.680 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:43.706 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T23:21:43.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:43.707 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:43.754 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:21:43.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:43.754 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:43.760 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:43.775 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:43.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:43.786 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:43.787 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:43.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:43.788 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:43.838 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:21:43.838 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T23:21:43.840 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:43.856 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:43.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:43.870 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:43.871 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:43.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:43.872 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:43.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2023-10-07T23:21:43.924+0000 7f2cca582380 -1 init, newargv = 0x564ee2cc2670 newargc=15 2023-10-07T23:21:43.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39356]: starting ceph client 2023-10-07T23:21:43.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39356]: starting fuse 2023-10-07T23:21:44.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:44.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:44.999 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:45.014 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:45.014 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:45.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:45.015 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:45.066 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:21:45.066 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-10-07T23:21:45.066 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:21:45.067 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2023-10-07T23:21:45.068 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-10-07T23:21:45.150 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.39356.asok 2023-10-07T23:21:45.336 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39356.asok 2023-10-07T23:21:45.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:45.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39356.asok status 2023-10-07T23:21:45.427 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-10-07T23:21:45.427 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-10-07T23:21:45.428 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:21:45.428 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:21:45.428 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2023-10-07T23:21:45.428 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-10-07T23:21:45.428 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T23:21:45.429 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "39356", 2023-10-07T23:21:45.429 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-10-07T23:21:45.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:45.429 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-10-07T23:21:45.429 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-10-07T23:21:45.430 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4693, 2023-10-07T23:21:45.430 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-10-07T23:21:45.430 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-10-07T23:21:45.430 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-10-07T23:21:45.430 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4693 2023-10-07T23:21:45.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:45.431 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:21:45.431 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:21:45.431 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:21:45.431 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3634962011 2023-10-07T23:21:45.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-10-07T23:21:45.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:45.432 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:21:45.432 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:21:45.432 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:21:45.433 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3634962011 2023-10-07T23:21:45.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:45.433 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4693 192.168.0.1:0/3634962011", 2023-10-07T23:21:45.433 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3634962011", 2023-10-07T23:21:45.433 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 6, 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 23, 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-10-07T23:21:45.434 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-10-07T23:21:45.436 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:21:45.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 Processor -- start 2023-10-07T23:21:45.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 -- start start 2023-10-07T23:21:45.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1c0156da0 0x7fe1c006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cb020 0x7fe1c00cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c00cc4e0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c0157220 con 0x7fe1c0156da0 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.546+0000 7fe1c4f40700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00cb480 con 0x7fe1c00cb020 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1c0156da0 0x7fe1c006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe1c0156da0 0x7fe1c006cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34440/0 (socket says 172.21.15.57:34440) 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 -- 172.21.15.57:0/389450208 learned_addr learned my addr 172.21.15.57:0/389450208 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:45.554 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1bffff700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cb020 0x7fe1c00cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 -- 172.21.15.57:0/389450208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cb020 msgr2=0x7fe1c00cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cb020 0x7fe1c00cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 -- 172.21.15.57:0/389450208 >> v1:172.21.15.154:6789/0 conn(0x7fe1c006c9a0 legacy=0x7fe1c0156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 -- 172.21.15.57:0/389450208 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c00ae7d0 con 0x7fe1c0156da0 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1beffd700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c0156da0 0x7fe1c006cd70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c9869d20270046 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1be7fc700 1 -- 172.21.15.57:0/389450208 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400b820 con 0x7fe1c0156da0 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1be7fc700 1 -- 172.21.15.57:0/389450208 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b400b980 con 0x7fe1c0156da0 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1be7fc700 1 -- 172.21.15.57:0/389450208 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400bd20 con 0x7fe1c0156da0 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 -- 172.21.15.57:0/389450208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c0156da0 msgr2=0x7fe1c006cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c0156da0 0x7fe1c006cd70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4009250 comp rx=0 tx=0).stop 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 -- 172.21.15.57:0/389450208 shutdown_connections 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c00cb020 0x7fe1c00cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.556 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/389450208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c0156da0 0x7fe1c006cd70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 -- 172.21.15.57:0/389450208 >> 172.21.15.57:0/389450208 conn(0x7fe1c00692f0 msgr2=0x7fe1c00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 -- 172.21.15.57:0/389450208 shutdown_connections 2023-10-07T23:21:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.550+0000 7fe1c4f40700 1 -- 172.21.15.57:0/389450208 wait complete. 2023-10-07T23:21:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 Processor -- start 2023-10-07T23:21:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- start start 2023-10-07T23:21:45.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c00cb020 0x7fe1c01593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 0x7fe1c0147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00b0ca0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00c73c0 con 0x7fe1c00cb020 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c0147b90 con 0x7fe1c0156da0 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c00cb020 0x7fe1c01593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 0x7fe1c0147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34448/0 (socket says 172.21.15.57:34448) 2023-10-07T23:21:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 -- 172.21.15.57:0/2398172507 learned_addr learned my addr 172.21.15.57:0/2398172507 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 -- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 msgr2=0x7fe1c0147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 0x7fe1c0147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 -- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c00cb020 msgr2=0x7fe1c01593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c00cb020 0x7fe1c01593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 -- 172.21.15.57:0/2398172507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40094f0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bffff700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 0x7fe1c0147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1bf7fe700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c4e0 tx=0x7fe1b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0011590 con 0x7fe1c006c9a0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b00116f0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c00b0950 con 0x7fe1c006c9a0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0011ab0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.554+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c0159d10 con 0x7fe1c006c9a0 2023-10-07T23:21:45.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.558+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c00711c0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.558+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe1b0012420 con 0x7fe1c006c9a0 2023-10-07T23:21:45.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.558+0000 7fe1a77fe700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 0x7fe1ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:45.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.558+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe1b007c060 con 0x7fe1c006c9a0 2023-10-07T23:21:45.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.562+0000 7fe1beffd700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 0x7fe1ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:45.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.562+0000 7fe1beffd700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 0x7fe1ac062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:45.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.566+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe1b004c5b0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.762+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1c00cb9a0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.762+0000 7fe1a77fe700 1 -- 172.21.15.57:0/2398172507 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe1b004fbd0 con 0x7fe1c006c9a0 2023-10-07T23:21:45.768 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:21:45.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 msgr2=0x7fe1ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 0x7fe1ac062a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4016000 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 msgr2=0x7fe1c0155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c4e0 tx=0x7fe1b000c7f0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 shutdown_connections 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe1c0156da0 0x7fe1c0147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe1ac060550 0x7fe1ac062a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe1c00cb020 0x7fe1c01593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 --2- 172.21.15.57:0/2398172507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe1c006c9a0 0x7fe1c0155db0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 >> 172.21.15.57:0/2398172507 conn(0x7fe1c00692f0 msgr2=0x7fe1c00696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 shutdown_connections 2023-10-07T23:21:45.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:45.766+0000 7fe1c4f40700 1 -- 172.21.15.57:0/2398172507 wait complete. 2023-10-07T23:21:45.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:21:45.788 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-07T23:21:45.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.789 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:45.797 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42516sec preferred_lft 42516sec 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:9b:82:8d:1f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:21:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::4a5:81ff:fe8f:3c66/64 scope link 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::74ca:eaff:feca:a10e/64 scope link 2023-10-07T23:21:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:45.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.800 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:21:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-07T23:21:45.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.857 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-07T23:21:45.864 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-07T23:21:45.865 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:21:45.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.865 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:45.865 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:45.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:21:45.866 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:21:45.866 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:21:45.866 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:45.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.905 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:21:45.911 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T23:21:45.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.912 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-10-07T23:21:45.917 INFO:teuthology.orchestra.run.smithi154.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-07T23:21:45.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.917 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:45.917 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:45.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T23:21:45.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-07T23:21:45.917 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:45.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:45.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-07T23:21:46.006 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:46.006 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 22:e0:bd:b4:06:66 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::20e0:bdff:feb4:666/64 scope link 2023-10-07T23:21:46.007 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:46.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-07T23:21:46.022 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:21:46.022 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 22:e0:bd:b4:06:66 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::20e0:bdff:feb4:666/64 scope link 2023-10-07T23:21:46.023 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:21:46.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-07T23:21:46.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-07T23:21:46.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-07T23:21:46.026 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:46.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:46.132 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:21:46.132 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:21:46.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2023-10-07T23:21:46.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.1 master ceph-brx 2023-10-07T23:21:46.132 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:21:46.160 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-07T23:21:46.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:46.163 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:46.170 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-07T23:21:46.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:21:46.170 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:46.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:46.233 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:46.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:46.249 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:46.250 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:46.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:46.250 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:21:46.257 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2023-10-07T23:21:46.257 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-07T23:21:46.301 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:46.318 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:46.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:46.329 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:46.330 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:46.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:46.330 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:46.378 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:21:46.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:2023-10-07T23:21:46.379+0000 7f2b5e220380 -1 init, newargv = 0x5577bceae390 newargc=15 2023-10-07T23:21:46.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[39469]: starting ceph client 2023-10-07T23:21:46.392 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi154.stderr:ceph-fuse[39469]: starting fuse 2023-10-07T23:21:47.480 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:21:47.499 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:21:47.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:21:47.509 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:21:47.510 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:21:47.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:47.511 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:21:47.562 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:21:47.562 INFO:teuthology.orchestra.run.smithi154.stdout:53 2023-10-07T23:21:47.562 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2023-10-07T23:21:47.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-10-07T23:21:47.563 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:21:47.564 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.3")) 2023-10-07T23:21:47.565 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-10-07T23:21:47.647 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.3.39469.asok 2023-10-07T23:21:47.834 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39469.asok 2023-10-07T23:21:47.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:47.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39469.asok status 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "3", 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-07T23:21:47.929 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "39469", 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4705, 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-10-07T23:21:47.930 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-10-07T23:21:47.931 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-10-07T23:21:47.931 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4705 2023-10-07T23:21:47.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1960012889 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-10-07T23:21:47.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1960012889 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:21:47.933 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4705 192.168.0.2:0/1960012889", 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.2:0/1960012889", 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 6, 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 23, 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-10-07T23:21:47.934 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-10-07T23:21:47.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:47.938 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-07T23:21:47.939 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-07T23:21:47.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:47.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:47.951 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:48.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.013 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-07T23:21:48.013 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:48.021 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-07T23:21:48.021 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:48.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.021 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:48.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.089 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-10-07T23:21:48.089 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:48.098 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-10-07T23:21:48.098 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:48.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:48.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.112 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-10-07T23:21:48.112 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:48.165 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-10-07T23:21:48.165 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:48.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:21:48.165 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:48.178 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-07T23:21:48.519 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:48.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 Processor -- start 2023-10-07T23:21:48.634 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 -- start start 2023-10-07T23:21:48.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f49e40bee60 0x7f49e40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40cba50 0x7f49e40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f49e40c6c10 0x7f49e40c7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e40cac40 con 0x7f49e40bee60 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e40cc3e0 con 0x7f49e40c6c10 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49eaefa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40cb610 con 0x7f49e40cba50 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f49e40bee60 0x7f49e40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f49e40bee60 0x7f49e40cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50530/0 (socket says 172.21.15.57:50530) 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 -- 172.21.15.57:0/991737394 learned_addr learned my addr 172.21.15.57:0/991737394 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e3fff700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40cba50 0x7f49e40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e9497700 1 --2- 172.21.15.57:0/991737394 >> v2:172.21.15.154:3300/0 conn(0x7f49e40c6c10 0x7f49e40c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 -- 172.21.15.57:0/991737394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40cba50 msgr2=0x7f49e40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:48.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40cba50 0x7f49e40cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 -- 172.21.15.57:0/991737394 >> v2:172.21.15.154:3300/0 conn(0x7f49e40c6c10 msgr2=0x7f49e40c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:48.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 --2- 172.21.15.57:0/991737394 >> v2:172.21.15.154:3300/0 conn(0x7f49e40c6c10 0x7f49e40c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 -- 172.21.15.57:0/991737394 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e40ca030 con 0x7f49e40bee60 2023-10-07T23:21:48.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e8c96700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40bee60 0x7f49e40cb0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c83f0 tx=0x7f49cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3309b05ab66ba9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e37fe700 1 -- 172.21.15.57:0/991737394 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00b640 con 0x7f49e40bee60 2023-10-07T23:21:48.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e37fe700 1 -- 172.21.15.57:0/991737394 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00b7a0 con 0x7f49e40bee60 2023-10-07T23:21:48.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.630+0000 7f49e37fe700 1 -- 172.21.15.57:0/991737394 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00bb60 con 0x7f49e40bee60 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- 172.21.15.57:0/991737394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40bee60 msgr2=0x7f49e40cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40bee60 0x7f49e40cb0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49e40c83f0 tx=0x7f49cc009250 comp rx=0 tx=0).stop 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- 172.21.15.57:0/991737394 shutdown_connections 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 --2- 172.21.15.57:0/991737394 >> v2:172.21.15.154:3300/0 conn(0x7f49e40c6c10 0x7f49e40c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40cba50 0x7f49e40cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 --2- 172.21.15.57:0/991737394 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40bee60 0x7f49e40cb0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- 172.21.15.57:0/991737394 >> 172.21.15.57:0/991737394 conn(0x7f49e40670e0 msgr2=0x7f49e40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:48.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- 172.21.15.57:0/991737394 shutdown_connections 2023-10-07T23:21:48.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- 172.21.15.57:0/991737394 wait complete. 2023-10-07T23:21:48.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 Processor -- start 2023-10-07T23:21:48.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.634+0000 7f49eaefa700 1 -- start start 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40c6c10 0x7f49e415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 0x7f49e40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40c2f80 con 0x7f49e40cba50 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e415e680 con 0x7f49e40bee60 2023-10-07T23:21:48.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e415e960 con 0x7f49e40c6c10 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e8c96700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 0x7f49e40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e8c96700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:55422/0 (socket says 172.21.15.57:55422) 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e8c96700 1 -- 172.21.15.57:0/3371934576 learned_addr learned my addr 172.21.15.57:0/3371934576 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e3fff700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40c6c10 0x7f49e415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 -- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40c6c10 msgr2=0x7f49e415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40c6c10 0x7f49e415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 -- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 msgr2=0x7f49e415c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 -- 172.21.15.57:0/3371934576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094f0 con 0x7f49e40cba50 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e8c96700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e9497700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 0x7f49e40c3410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e950 tx=0x7f49d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4013680 con 0x7f49e40cba50 2023-10-07T23:21:48.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4013800 con 0x7f49e40cba50 2023-10-07T23:21:48.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4013bc0 con 0x7f49e40cba50 2023-10-07T23:21:48.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e415ec40 con 0x7f49e40cba50 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.638+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e415e2e0 con 0x7f49e40cba50 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f49d400c3e0 con 0x7f49e40cba50 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e40c3130 con 0x7f49e40cba50 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49e0ff9700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 0x7f49c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49d407c680 con 0x7f49e40cba50 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49e8c96700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 0x7f49c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.642+0000 7f49e8c96700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 0x7f49c8062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49cc003080 tx=0x7f49cc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.650+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f49e40c3130 con 0x7f49e40cba50 2023-10-07T23:21:48.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:21:48.698 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:21:48.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.810+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49e40711c0 con 0x7f49e40cba50 2023-10-07T23:21:48.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.814+0000 7f49e0ff9700 1 -- 172.21.15.57:0/3371934576 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f49d4050700 con 0x7f49e40cba50 2023-10-07T23:21:48.819 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:48.820 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:48.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.814+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 msgr2=0x7f49c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.814+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 0x7f49c8062a00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49cc003080 tx=0x7f49cc017000 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.814+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 msgr2=0x7f49e40c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.814+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 0x7f49e40c3410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e950 tx=0x7f49d40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 shutdown_connections 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f49e40cba50 0x7f49e40c3410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f49e40c6c10 0x7f49e415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f49c8060550 0x7f49c8062a00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 --2- 172.21.15.57:0/3371934576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f49e40bee60 0x7f49e415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 >> 172.21.15.57:0/3371934576 conn(0x7f49e40670e0 msgr2=0x7f49e415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:48.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 shutdown_connections 2023-10-07T23:21:48.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.818+0000 7f49eaefa700 1 -- 172.21.15.57:0/3371934576 wait complete. 2023-10-07T23:21:48.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:48.834 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:48.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 Processor -- start 2023-10-07T23:21:48.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 -- start start 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 0x7f3c980aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9806a900 con 0x7f3c9806c800 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9806a600 con 0x7f3c980ca600 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c9e011700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980b0450 con 0x7f3c98159030 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c977fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 0x7f3c980aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c96ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3c9806c800 0x7f3c9806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48224/0 (socket says 172.21.15.57:48224) 2023-10-07T23:21:48.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c96ffd700 1 -- 172.21.15.57:0/2445613517 learned_addr learned my addr 172.21.15.57:0/2445613517 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:48.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737896186 0 0) 0x7f3c9806a900 con 0x7f3c9806c800 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c80003570 con 0x7f3c9806c800 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683224420 0 0) 0x7f3c9806a600 con 0x7f3c980ca600 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9806a900 con 0x7f3c980ca600 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c977fe700 1 -- 172.21.15.57:0/2445613517 >> v1:172.21.15.154:6789/0 conn(0x7f3c980ca600 legacy=0x7f3c980caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c977fe700 1 -- 172.21.15.57:0/2445613517 >> v1:172.21.15.57:6789/0 conn(0x7f3c9806c800 legacy=0x7f3c9806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c977fe700 1 -- 172.21.15.57:0/2445613517 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980cb350 con 0x7f3c98159030 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.946+0000 7f3c977fe700 1 --2- 172.21.15.57:0/2445613517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 0x7f3c980aff10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e90 tx=0x7f3c90009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd327520c29aa21 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000e070 con 0x7f3c98159030 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9000bb80 con 0x7f3c98159030 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c967fc700 1 -- 172.21.15.57:0/2445613517 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90012430 con 0x7f3c98159030 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- 172.21.15.57:0/2445613517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 msgr2=0x7f3c980aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 --2- 172.21.15.57:0/2445613517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 0x7f3c980aff10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e90 tx=0x7f3c90009780 comp rx=0 tx=0).stop 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- 172.21.15.57:0/2445613517 shutdown_connections 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 --2- 172.21.15.57:0/2445613517 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c98159030 0x7f3c980aff10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- 172.21.15.57:0/2445613517 >> 172.21.15.57:0/2445613517 conn(0x7f3c98068650 msgr2=0x7f3c98068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- 172.21.15.57:0/2445613517 shutdown_connections 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- 172.21.15.57:0/2445613517 wait complete. 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 Processor -- start 2023-10-07T23:21:48.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- start start 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 0x7f3c98150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 0x7f3c980c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c4ab0 con 0x7f3c98159030 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98150d50 con 0x7f3c9806c800 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c9e011700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980c3ea0 con 0x7f3c980ca600 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c97fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c977fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 0x7f3c98150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c97fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50542/0 (socket says 172.21.15.57:50542) 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c97fff700 1 -- 172.21.15.57:0/1930241759 learned_addr learned my addr 172.21.15.57:0/1930241759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.950+0000 7f3c96ffd700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 0x7f3c980c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 -- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 msgr2=0x7f3c980c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 0x7f3c980c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 -- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 msgr2=0x7f3c98150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 0x7f3c98150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 -- 172.21.15.57:0/1930241759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90009a20 con 0x7f3c98159030 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c977fe700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 0x7f3c98150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:48.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c96ffd700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 0x7f3c980c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c97fff700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c590 tx=0x7f3c88007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88016070 con 0x7f3c98159030 2023-10-07T23:21:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c980c4180 con 0x7f3c98159030 2023-10-07T23:21:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c88011660 con 0x7f3c98159030 2023-10-07T23:21:48.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88011a40 con 0x7f3c98159030 2023-10-07T23:21:48.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c980c4740 con 0x7f3c98159030 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c980c7000 con 0x7f3c98159030 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.954+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3c88012420 con 0x7f3c98159030 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.958+0000 7f3c7f7fe700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 0x7f3c84062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.958+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c8807c530 con 0x7f3c98159030 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.958+0000 7f3c977fe700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 0x7f3c84062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:48.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.958+0000 7f3c977fe700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 0x7f3c84062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009de0 tx=0x7f3c90019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:48.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:48.962+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3c980c7000 con 0x7f3c98159030 2023-10-07T23:21:49.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.126+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c980c7000 con 0x7f3c98159030 2023-10-07T23:21:49.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.126+0000 7f3c7f7fe700 1 -- 172.21.15.57:0/1930241759 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c980c7000 con 0x7f3c98159030 2023-10-07T23:21:49.132 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 msgr2=0x7f3c84062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 0x7f3c84062a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009de0 tx=0x7f3c90019000 comp rx=0 tx=0).stop 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 msgr2=0x7f3c98152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c590 tx=0x7f3c88007d80 comp rx=0 tx=0).stop 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 shutdown_connections 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3c98159030 0x7f3c98152f80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3c980ca600 0x7f3c980c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3c84060550 0x7f3c84062a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 --2- 172.21.15.57:0/1930241759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c9806c800 0x7f3c98150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 >> 172.21.15.57:0/1930241759 conn(0x7f3c98068650 msgr2=0x7f3c980caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 shutdown_connections 2023-10-07T23:21:49.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.130+0000 7f3c9e011700 1 -- 172.21.15.57:0/1930241759 wait complete. 2023-10-07T23:21:49.147 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:49.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 Processor -- start 2023-10-07T23:21:49.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 -- start start 2023-10-07T23:21:49.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ccc0ca600 0x7f8ccc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc06a900 con 0x7f8ccc06c800 2023-10-07T23:21:49.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0b0450 con 0x7f8ccc0ca600 2023-10-07T23:21:49.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd35f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc06a600 con 0x7f8ccc159030 2023-10-07T23:21:49.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ccc0ca600 0x7f8ccc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ccc0ca600 0x7f8ccc0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50558/0 (socket says 172.21.15.57:50558) 2023-10-07T23:21:49.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 -- 172.21.15.57:0/2218723123 learned_addr learned my addr 172.21.15.57:0/2218723123 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:49.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd0b8d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8ccc06c800 0x7f8ccc06cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48238/0 (socket says 172.21.15.57:48238) 2023-10-07T23:21:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd138e700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 -- 172.21.15.57:0/2218723123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 msgr2=0x7f8ccc0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 -- 172.21.15.57:0/2218723123 >> v1:172.21.15.57:6789/0 conn(0x7f8ccc06c800 legacy=0x7f8ccc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 -- 172.21.15.57:0/2218723123 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0cb350 con 0x7f8ccc0ca600 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cd1b8f700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc0caa80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009f20 tx=0x7f8cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434395db0739ade6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cc3fff700 1 -- 172.21.15.57:0/2218723123 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400e070 con 0x7f8ccc0ca600 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cc3fff700 1 -- 172.21.15.57:0/2218723123 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400ba80 con 0x7f8ccc0ca600 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.254+0000 7f8cc3fff700 1 -- 172.21.15.57:0/2218723123 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40092e0 con 0x7f8ccc0ca600 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2218723123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc0ca600 msgr2=0x7f8ccc0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc0caa80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009f20 tx=0x7f8cc4009840 comp rx=0 tx=0).stop 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2218723123 shutdown_connections 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc0caa80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2218723123 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2218723123 >> 172.21.15.57:0/2218723123 conn(0x7f8ccc068650 msgr2=0x7f8ccc068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2218723123 shutdown_connections 2023-10-07T23:21:49.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2218723123 wait complete. 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 Processor -- start 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- start start 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc14b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc14f890 con 0x7f8ccc06c800 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc14f590 con 0x7f8ccc0ca600 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd35f2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c7900 con 0x7f8ccc159030 2023-10-07T23:21:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50564/0 (socket says 172.21.15.57:50564) 2023-10-07T23:21:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 -- 172.21.15.57:0/2038325707 learned_addr learned my addr 172.21.15.57:0/2038325707 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd0b8d700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc14b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd1b8f700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 -- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 msgr2=0x7f8ccc0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 -- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 msgr2=0x7f8ccc14b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.258+0000 7f8cd138e700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc14b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd138e700 1 -- 172.21.15.57:0/2038325707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009ae0 con 0x7f8ccc06c800 2023-10-07T23:21:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd0b8d700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc14b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd138e700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d890 tx=0x7f8cc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80125f0 con 0x7f8ccc06c800 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8012750 con 0x7f8ccc06c800 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc0c7be0 con 0x7f8ccc06c800 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8012b10 con 0x7f8ccc06c800 2023-10-07T23:21:49.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc0c7ec0 con 0x7f8ccc06c800 2023-10-07T23:21:49.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc0647b0 con 0x7f8ccc06c800 2023-10-07T23:21:49.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8cc801e460 con 0x7f8ccc06c800 2023-10-07T23:21:49.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 0x7f8cb0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8cc807cc30 con 0x7f8ccc06c800 2023-10-07T23:21:49.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.262+0000 7f8cd0b8d700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 0x7f8cb0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.266+0000 7f8cd0b8d700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 0x7f8cb0062a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc16c830 tx=0x7f8cbc007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.270+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8cc8050cb0 con 0x7f8ccc06c800 2023-10-07T23:21:49.436 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.430+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ccc0cb630 con 0x7f8ccc06c800 2023-10-07T23:21:49.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.430+0000 7f8cc17fa700 1 -- 172.21.15.57:0/2038325707 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8cc8050840 con 0x7f8ccc06c800 2023-10-07T23:21:49.437 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 msgr2=0x7f8cb0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 0x7f8cb0062a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc16c830 tx=0x7f8cbc007eb0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 msgr2=0x7f8ccc0c3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d890 tx=0x7f8cc8008040 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 shutdown_connections 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ccc159030 0x7f8ccc0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8cb0060580 0x7f8cb0062a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ccc0ca600 0x7f8ccc14b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 --2- 172.21.15.57:0/2038325707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ccc06c800 0x7f8ccc0c3730 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 >> 172.21.15.57:0/2038325707 conn(0x7f8ccc068650 msgr2=0x7f8ccc0ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 shutdown_connections 2023-10-07T23:21:49.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.434+0000 7f8cd35f2700 1 -- 172.21.15.57:0/2038325707 wait complete. 2023-10-07T23:21:49.452 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:49.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.562+0000 7f2f0b59e700 1 Processor -- start 2023-10-07T23:21:49.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.562+0000 7f2f0b59e700 1 -- start start 2023-10-07T23:21:49.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0be7f0 0x7f2f0c0bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0c0c9fc0 con 0x7f2f0c0c8580 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c0cc820 con 0x7f2f0c0c8950 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0abe00 con 0x7f2f0c0be7f0 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50568/0 (socket says 172.21.15.57:50568) 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2415632694 learned_addr learned my addr 172.21.15.57:0/2415632694 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:49.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0ad9d700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0be7f0 0x7f2f0c0bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2415632694 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0be7f0 msgr2=0x7f2f0c0bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:49.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0be7f0 0x7f2f0c0bec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2415632694 >> v1:172.21.15.154:6789/0 conn(0x7f2f0c0c8580 legacy=0x7f2f0c0cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2415632694 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c15d850 con 0x7f2f0c0c8950 2023-10-07T23:21:49.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f09d9b700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e36089ec22ddd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0959a700 1 -- 172.21.15.57:0/2415632694 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc009ea0 con 0x7f2f0c0c8950 2023-10-07T23:21:49.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0959a700 1 -- 172.21.15.57:0/2415632694 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00b860 con 0x7f2f0c0c8950 2023-10-07T23:21:49.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0959a700 1 -- 172.21.15.57:0/2415632694 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00bc00 con 0x7f2f0c0c8950 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2415632694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8950 msgr2=0x7f2f0c0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0099b0 tx=0x7f2efc009250 comp rx=0 tx=0).stop 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2415632694 shutdown_connections 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0be7f0 0x7f2f0c0bec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2415632694 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8950 0x7f2f0c0ab8c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2415632694 >> 172.21.15.57:0/2415632694 conn(0x7f2f0c0670e0 msgr2=0x7f2f0c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2415632694 shutdown_connections 2023-10-07T23:21:49.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.566+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2415632694 wait complete. 2023-10-07T23:21:49.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.570+0000 7f2f0b59e700 1 Processor -- start 2023-10-07T23:21:49.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.570+0000 7f2f0b59e700 1 -- start start 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c0be7f0 0x7f2f0c16da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0c8950 0x7f2f0c170920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c170e60 con 0x7f2f0c0c8580 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0c4fd0 con 0x7f2f0c0be7f0 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0af910 con 0x7f2f0c0c8950 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50572/0 (socket says 172.21.15.57:50572) 2023-10-07T23:21:49.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2696219981 learned_addr learned my addr 172.21.15.57:0/2696219981 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:49.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0ad9d700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0c8950 0x7f2f0c170920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0a59c700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c0be7f0 0x7f2f0c16da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0c8950 msgr2=0x7f2f0c170920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0c8950 0x7f2f0c170920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c0be7f0 msgr2=0x7f2f0c16da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c0be7f0 0x7f2f0c16da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 -- 172.21.15.57:0/2696219981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0094f0 con 0x7f2f0c0c8580 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f09d9b700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2efc006fd0 tx=0x7f2efc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00bbe0 con 0x7f2f0c0c8580 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc025440 con 0x7f2f0c0c8580 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc025800 con 0x7f2f0c0c8580 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c0c4cf0 con 0x7f2f0c0c8580 2023-10-07T23:21:49.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c0af5a0 con 0x7f2f0c0c8580 2023-10-07T23:21:49.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.574+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0c067d60 con 0x7f2f0c0c8580 2023-10-07T23:21:49.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.578+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2efc02e440 con 0x7f2f0c0c8580 2023-10-07T23:21:49.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.578+0000 7f2efa7fc700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 0x7f2eec062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.578+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2efc07e420 con 0x7f2f0c0c8580 2023-10-07T23:21:49.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.578+0000 7f2f0a59c700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 0x7f2eec062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.578+0000 7f2f0a59c700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 0x7f2eec062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2f00003a80 tx=0x7f2f00007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.582+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f0c067d60 con 0x7f2f0c0c8580 2023-10-07T23:21:49.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.742+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f0c067d60 con 0x7f2f0c0c8580 2023-10-07T23:21:49.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2efa7fc700 1 -- 172.21.15.57:0/2696219981 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f0c067d60 con 0x7f2f0c0c8580 2023-10-07T23:21:49.750 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 msgr2=0x7f2eec062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 0x7f2eec062a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2f00003a80 tx=0x7f2f00007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 msgr2=0x7f2f0c0c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2efc006fd0 tx=0x7f2efc017000 comp rx=0 tx=0).stop 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 shutdown_connections 2023-10-07T23:21:49.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2f0c0c8950 0x7f2f0c170920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f0c0c8580 0x7f2f0c0c3430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eec060550 0x7f2eec062a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 --2- 172.21.15.57:0/2696219981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f0c0be7f0 0x7f2f0c16da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 >> 172.21.15.57:0/2696219981 conn(0x7f2f0c0670e0 msgr2=0x7f2f0c06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 shutdown_connections 2023-10-07T23:21:49.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.746+0000 7f2f0b59e700 1 -- 172.21.15.57:0/2696219981 wait complete. 2023-10-07T23:21:49.764 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.866+0000 7f113359e700 1 Processor -- start 2023-10-07T23:21:49.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.866+0000 7f113359e700 1 -- start start 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f113359e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1134150150 0x7f113406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f113359e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11341594f0 con 0x7f11340c93c0 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f113359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1134159790 con 0x7f113414fd80 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f113359e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1134159a70 con 0x7f1134150150 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1132d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f11340c93c0 0x7f113406af70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48254/0 (socket says 172.21.15.57:48254) 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f112bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1134150150 0x7f113406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1132d9d700 1 -- 172.21.15.57:0/4071378284 learned_addr learned my addr 172.21.15.57:0/4071378284 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:49.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095573868 0 0) 0x7f11341594f0 con 0x7f11340c93c0 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1120003570 con 0x7f11340c93c0 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472213037 0 0) 0x7f1134159790 con 0x7f113414fd80 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11341594f0 con 0x7f113414fd80 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f112bfff700 1 -- 172.21.15.57:0/4071378284 >> v1:172.21.15.154:6789/0 conn(0x7f113414fd80 legacy=0x7f11340abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f112bfff700 1 -- 172.21.15.57:0/4071378284 >> v1:172.21.15.57:6789/0 conn(0x7f11340c93c0 legacy=0x7f113406af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f112bfff700 1 -- 172.21.15.57:0/4071378284 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134159d50 con 0x7f1134150150 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f112bfff700 1 --2- 172.21.15.57:0/4071378284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f113406c800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f111c0098e0 tx=0x7f111c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb8a1f45c3f4f57 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011630 con 0x7f1134150150 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c011790 con 0x7f1134150150 2023-10-07T23:21:49.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.870+0000 7f1131d9b700 1 -- 172.21.15.57:0/4071378284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c011b30 con 0x7f1134150150 2023-10-07T23:21:49.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- 172.21.15.57:0/4071378284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 msgr2=0x7f113406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 --2- 172.21.15.57:0/4071378284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f113406c800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f111c0098e0 tx=0x7f111c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- 172.21.15.57:0/4071378284 shutdown_connections 2023-10-07T23:21:49.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 --2- 172.21.15.57:0/4071378284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f113406c800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- 172.21.15.57:0/4071378284 >> 172.21.15.57:0/4071378284 conn(0x7f1134067b80 msgr2=0x7f11340c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:49.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- 172.21.15.57:0/4071378284 shutdown_connections 2023-10-07T23:21:49.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- 172.21.15.57:0/4071378284 wait complete. 2023-10-07T23:21:49.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 Processor -- start 2023-10-07T23:21:49.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 -- start start 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.874+0000 7f113359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11340c93c0 0x7f11341691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113414fd80 0x7f113413dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340a8950 con 0x7f1134150150 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341697c0 con 0x7f113414fd80 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1134169aa0 con 0x7f11340c93c0 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50592/0 (socket says 172.21.15.57:50592) 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f112bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113414fd80 0x7f113413dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 -- 172.21.15.57:0/2227499334 learned_addr learned my addr 172.21.15.57:0/2227499334 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:49.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113259c700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11340c93c0 0x7f11341691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 -- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11340c93c0 msgr2=0x7f11341691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11340c93c0 0x7f11341691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 -- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113414fd80 msgr2=0x7f113413dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113414fd80 0x7f113413dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 -- 172.21.15.57:0/2227499334 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c009420 con 0x7f1134150150 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f1132d9d700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f112c00cd00 tx=0x7f112c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ce00 con 0x7f1134150150 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00aac0 con 0x7f1134150150 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1134169d80 con 0x7f1134150150 2023-10-07T23:21:49.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00ae80 con 0x7f1134150150 2023-10-07T23:21:49.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113413e130 con 0x7f1134150150 2023-10-07T23:21:49.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.878+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11340a8ac0 con 0x7f1134150150 2023-10-07T23:21:49.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.882+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f112c007300 con 0x7f1134150150 2023-10-07T23:21:49.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.882+0000 7f112a7fc700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 0x7f1114062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:49.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.882+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f112c07cad0 con 0x7f1134150150 2023-10-07T23:21:49.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.882+0000 7f113259c700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 0x7f1114062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:49.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.882+0000 7f113259c700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 0x7f1114062a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f11340cbab0 tx=0x7f1124007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:49.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:49.886+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f11340a8ac0 con 0x7f1134150150 2023-10-07T23:21:50.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.050+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11340a8a90 con 0x7f1134150150 2023-10-07T23:21:50.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f112a7fc700 1 -- 172.21.15.57:0/2227499334 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f112c050bd0 con 0x7f1134150150 2023-10-07T23:21:50.058 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:50.058 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 msgr2=0x7f1114062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 0x7f1114062a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f11340cbab0 tx=0x7f1124007d50 comp rx=0 tx=0).stop 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 msgr2=0x7f11340af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f112c00cd00 tx=0x7f112c001ac0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 shutdown_connections 2023-10-07T23:21:50.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1134150150 0x7f11340af050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1114060580 0x7f1114062a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f113414fd80 0x7f113413dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 --2- 172.21.15.57:0/2227499334 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f11340c93c0 0x7f11341691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 >> 172.21.15.57:0/2227499334 conn(0x7f1134067b80 msgr2=0x7f113406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 shutdown_connections 2023-10-07T23:21:50.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.054+0000 7f113359e700 1 -- 172.21.15.57:0/2227499334 wait complete. 2023-10-07T23:21:50.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:50.073 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:50.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 Processor -- start 2023-10-07T23:21:50.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 -- start start 2023-10-07T23:21:50.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa2d80ab740 0x7fa2d80c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80abe00 con 0x7fa2d815a020 2023-10-07T23:21:50.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d80ae830 con 0x7fa2d8155a70 2023-10-07T23:21:50.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2ddbb5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d80c9de0 con 0x7fa2d80ab740 2023-10-07T23:21:50.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.182+0000 7fa2d6ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa2d80ab740 0x7fa2d80c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa2d80ab740 0x7fa2d80c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50608/0 (socket says 172.21.15.57:50608) 2023-10-07T23:21:50.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d7fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa2d815a020 0x7fa2d806c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48260/0 (socket says 172.21.15.57:48260) 2023-10-07T23:21:50.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/148318082 learned_addr learned my addr 172.21.15.57:0/148318082 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d67fc700 1 -- 172.21.15.57:0/148318082 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440201162 0 0) 0x7fa2d80abe00 con 0x7fa2d815a020 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d67fc700 1 -- 172.21.15.57:0/148318082 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c0003570 con 0x7fa2d815a020 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/148318082 >> v1:172.21.15.154:6789/0 conn(0x7fa2d8155a70 legacy=0x7fa2d8159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/148318082 >> v1:172.21.15.57:6789/0 conn(0x7fa2d815a020 legacy=0x7fa2d806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/148318082 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80abb90 con 0x7fa2d80ab740 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d6ffd700 1 --2- 172.21.15.57:0/148318082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c98a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0099b0 tx=0x7fa2cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe512ba20cce9b7a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d67fc700 1 -- 172.21.15.57:0/148318082 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc009ea0 con 0x7fa2d80ab740 2023-10-07T23:21:50.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d67fc700 1 -- 172.21.15.57:0/148318082 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00b860 con 0x7fa2d80ab740 2023-10-07T23:21:50.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2d67fc700 1 -- 172.21.15.57:0/148318082 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00bbe0 con 0x7fa2d80ab740 2023-10-07T23:21:50.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/148318082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d80ab740 msgr2=0x7fa2d80c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/148318082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c98a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0099b0 tx=0x7fa2cc009250 comp rx=0 tx=0).stop 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/148318082 shutdown_connections 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/148318082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c98a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/148318082 >> 172.21.15.57:0/148318082 conn(0x7fa2d80ccf10 msgr2=0x7fa2d80c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/148318082 shutdown_connections 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/148318082 wait complete. 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 Processor -- start 2023-10-07T23:21:50.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.186+0000 7fa2ddbb5700 1 -- start start 2023-10-07T23:21:50.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2d815a020 0x7fa2d8168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c3d80 con 0x7fa2d8155a70 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81685b0 con 0x7fa2d80ab740 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80c3450 con 0x7fa2d815a020 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50614/0 (socket says 172.21.15.57:50614) 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/3811595621 learned_addr learned my addr 172.21.15.57:0/3811595621 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d77fe700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d7fff700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2d815a020 0x7fa2d8168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2d815a020 msgr2=0x7fa2d8168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2d815a020 0x7fa2d8168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d80ab740 msgr2=0x7fa2d80c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 -- 172.21.15.57:0/3811595621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0094f0 con 0x7fa2d8155a70 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2d6ffd700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806a020 tx=0x7fa2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00bbe0 con 0x7fa2d8155a70 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc000d10 con 0x7fa2d8155a70 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d80c3730 con 0x7fa2d8155a70 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc011670 con 0x7fa2d8155a70 2023-10-07T23:21:50.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80c4750 con 0x7fa2d8155a70 2023-10-07T23:21:50.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.190+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d80c3f30 con 0x7fa2d8155a70 2023-10-07T23:21:50.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.194+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa2cc011a60 con 0x7fa2d8155a70 2023-10-07T23:21:50.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.194+0000 7fa2bf7fe700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 0x7fa2c4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.194+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2cc07ded0 con 0x7fa2d8155a70 2023-10-07T23:21:50.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.194+0000 7fa2d77fe700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 0x7fa2c4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.194+0000 7fa2d77fe700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 0x7fa2c4062a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2d81556d0 tx=0x7fa2c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.202+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa2d80c3f30 con 0x7fa2d8155a70 2023-10-07T23:21:50.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.362+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2d80647b0 con 0x7fa2d8155a70 2023-10-07T23:21:50.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.362+0000 7fa2bf7fe700 1 -- 172.21.15.57:0/3811595621 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2cc015020 con 0x7fa2d8155a70 2023-10-07T23:21:50.368 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 msgr2=0x7fa2c4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 0x7fa2c4062a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2d81556d0 tx=0x7fa2c8009040 comp rx=0 tx=0).stop 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 msgr2=0x7fa2d80c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806a020 tx=0x7fa2cc017000 comp rx=0 tx=0).stop 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 shutdown_connections 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa2d815a020 0x7fa2d8168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa2d8155a70 0x7fa2d80c4210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa2c4060580 0x7fa2c4062a30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 --2- 172.21.15.57:0/3811595621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2d80ab740 0x7fa2d80c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 >> 172.21.15.57:0/3811595621 conn(0x7fa2d80ccf10 msgr2=0x7fa2d80c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 shutdown_connections 2023-10-07T23:21:50.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.366+0000 7fa2ddbb5700 1 -- 172.21.15.57:0/3811595621 wait complete. 2023-10-07T23:21:50.382 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:50.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.490+0000 7f4be9f5f700 1 Processor -- start 2023-10-07T23:21:50.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.490+0000 7f4be9f5f700 1 -- start start 2023-10-07T23:21:50.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4be40ca600 0x7f4be40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be406a900 con 0x7f4be4159030 2023-10-07T23:21:50.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be40b0450 con 0x7f4be40ca600 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be406a600 con 0x7f4be406cf10 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4be40ca600 0x7f4be40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4be40ca600 0x7f4be40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:55476/0 (socket says 172.21.15.57:55476) 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 -- 172.21.15.57:0/2766475647 learned_addr learned my addr 172.21.15.57:0/2766475647 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be2ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:55946/0 (socket says 172.21.15.57:55946) 2023-10-07T23:21:50.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be37fe700 1 --1- 172.21.15.57:0/2766475647 >> v1:172.21.15.154:6789/0 conn(0x7f4be4159030 0x7f4be406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:56542/0 (socket says 172.21.15.57:56542) 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 -- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 msgr2=0x7f4be40aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 -- 172.21.15.57:0/2766475647 >> v1:172.21.15.154:6789/0 conn(0x7f4be4159030 legacy=0x7f4be406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 -- 172.21.15.57:0/2766475647 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40cb350 con 0x7f4be40ca600 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be2ffd700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be3fff700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be40caa80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009f20 tx=0x7f4bdc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ccad6a55e797dff server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be27fc700 1 -- 172.21.15.57:0/2766475647 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00e070 con 0x7f4be40ca600 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be27fc700 1 -- 172.21.15.57:0/2766475647 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00b9d0 con 0x7f4be40ca600 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be27fc700 1 -- 172.21.15.57:0/2766475647 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0092e0 con 0x7f4be40ca600 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 msgr2=0x7f4be40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be40caa80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009f20 tx=0x7f4bdc009840 comp rx=0 tx=0).stop 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- 172.21.15.57:0/2766475647 shutdown_connections 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be40caa80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/2766475647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be40aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.494+0000 7f4be9f5f700 1 -- 172.21.15.57:0/2766475647 >> 172.21.15.57:0/2766475647 conn(0x7f4be4068650 msgr2=0x7f4be4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- 172.21.15.57:0/2766475647 shutdown_connections 2023-10-07T23:21:50.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- 172.21.15.57:0/2766475647 wait complete. 2023-10-07T23:21:50.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 Processor -- start 2023-10-07T23:21:50.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- start start 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be4157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41561d0 con 0x7f4be4159030 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41564b0 con 0x7f4be40ca600 2023-10-07T23:21:50.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be9f5f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4156790 con 0x7f4be406cf10 2023-10-07T23:21:50.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50622/0 (socket says 172.21.15.57:50622) 2023-10-07T23:21:50.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be3fff700 1 -- 172.21.15.57:0/610936479 learned_addr learned my addr 172.21.15.57:0/610936479 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:50.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be37fe700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be4157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.498+0000 7f4be2ffd700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 -- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 msgr2=0x7f4be4157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be4157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 -- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 msgr2=0x7f4be4153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be4153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 -- 172.21.15.57:0/610936479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4009420 con 0x7f4be4159030 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be3fff700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b610 tx=0x7f4bdc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc008b80 con 0x7f4be4159030 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc008ce0 con 0x7f4be4159030 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc009ae0 con 0x7f4be4159030 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc026690 con 0x7f4be4159030 2023-10-07T23:21:50.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be415a160 con 0x7f4be4159030 2023-10-07T23:21:50.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be40647b0 con 0x7f4be4159030 2023-10-07T23:21:50.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4bdc003040 con 0x7f4be4159030 2023-10-07T23:21:50.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 0x7f4bc4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4bdc07e780 con 0x7f4be4159030 2023-10-07T23:21:50.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.502+0000 7f4be37fe700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 0x7f4bc4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.506+0000 7f4be37fe700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 0x7f4bc4062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc007fd0 tx=0x7f4bcc007d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.510+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4bdc04ed80 con 0x7f4be4159030 2023-10-07T23:21:50.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.670+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4be40cb630 con 0x7f4be4159030 2023-10-07T23:21:50.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.670+0000 7f4bdb7fe700 1 -- 172.21.15.57:0/610936479 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bdc0523a0 con 0x7f4be4159030 2023-10-07T23:21:50.677 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:50.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 msgr2=0x7f4bc4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 0x7f4bc4062a00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc007fd0 tx=0x7f4bcc007d70 comp rx=0 tx=0).stop 2023-10-07T23:21:50.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 msgr2=0x7f4be4153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b610 tx=0x7f4bdc018000 comp rx=0 tx=0).stop 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 shutdown_connections 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4be4159030 0x7f4be4153890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4bc4060550 0x7f4bc4062a00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4be40ca600 0x7f4be4153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 --2- 172.21.15.57:0/610936479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4be406cf10 0x7f4be4157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 >> 172.21.15.57:0/610936479 conn(0x7f4be4068650 msgr2=0x7f4be40b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 shutdown_connections 2023-10-07T23:21:50.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.674+0000 7f4be9f5f700 1 -- 172.21.15.57:0/610936479 wait complete. 2023-10-07T23:21:50.691 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:50.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.794+0000 7f0d8e085700 1 Processor -- start 2023-10-07T23:21:50.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.794+0000 7f0d8e085700 1 -- start start 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d880c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d880c9fc0 con 0x7f0d880ab8c0 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8815d850 con 0x7f0d880be7f0 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880cc820 con 0x7f0d880c8580 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8c882700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0d880ab8c0 0x7f0d880cb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48262/0 (socket says 172.21.15.57:48262) 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8c882700 1 -- 172.21.15.57:0/573166508 learned_addr learned my addr 172.21.15.57:0/573166508 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d083700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d880c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409712585 0 0) 0x7f0d880c9fc0 con 0x7f0d880ab8c0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d74003570 con 0x7f0d880ab8c0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969591870 0 0) 0x7f0d8815d850 con 0x7f0d880be7f0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d880c9fc0 con 0x7f0d880be7f0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462287553 0 0) 0x7f0d74003570 con 0x7f0d880ab8c0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 msgr2=0x7f0d880c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d880c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 >> v1:172.21.15.154:6789/0 conn(0x7f0d880be7f0 legacy=0x7f0d880bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880cbfd0 con 0x7f0d880ab8c0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f0d78001470 con 0x7f0d880ab8c0 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 >> v1:172.21.15.57:6789/0 conn(0x7f0d880ab8c0 legacy=0x7f0d880cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74003c20 0x7f0d74004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d74004da0 0x7f0d74005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880cc820 con 0x7f0d74004da0 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400bbf0 con 0x7f0d74003c20 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400bdb0 con 0x7f0d880c8580 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d74004da0 0x7f0d74005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d083700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74003c20 0x7f0d74004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8c882700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 -- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 msgr2=0x7f0d74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 -- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74003c20 msgr2=0x7f0d74004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:50.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74003c20 0x7f0d74004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 -- 172.21.15.57:0/573166508 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74003570 con 0x7f0d74004da0 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8c882700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d880c8580 0x7f0d74004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d74004da0 0x7f0d74005210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d80002120 tx=0x7f0d800017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e516f6dcf9bdda server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80001ab0 con 0x7f0d74004da0 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000c980 con 0x7f0d74004da0 2023-10-07T23:21:50.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d7ffff700 1 -- 172.21.15.57:0/573166508 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000cd20 con 0x7f0d74004da0 2023-10-07T23:21:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- 172.21.15.57:0/573166508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d74004da0 msgr2=0x7f0d74005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 --2- 172.21.15.57:0/573166508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d74004da0 0x7f0d74005210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d80002120 tx=0x7f0d800017a0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8d884700 1 -- 172.21.15.57:0/573166508 reap_dead start 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- 172.21.15.57:0/573166508 shutdown_connections 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- 172.21.15.57:0/573166508 >> 172.21.15.57:0/573166508 conn(0x7f0d880670e0 msgr2=0x7f0d880674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- 172.21.15.57:0/573166508 shutdown_connections 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.798+0000 7f0d8e085700 1 -- 172.21.15.57:0/573166508 wait complete. 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 Processor -- start 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- start start 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74004da0 0x7f0d880c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d8816dd10 0x7f0d880abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8816e760 con 0x7f0d880aeff0 2023-10-07T23:21:50.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8816e1a0 con 0x7f0d74004da0 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8816e480 con 0x7f0d8816dd10 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8d083700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74004da0 0x7f0d880c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50634/0 (socket says 172.21.15.57:50634) 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 -- 172.21.15.57:0/3629650990 learned_addr learned my addr 172.21.15.57:0/3629650990 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8d884700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d8816dd10 0x7f0d880abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 -- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d8816dd10 msgr2=0x7f0d880abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d8816dd10 0x7f0d880abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 -- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74004da0 msgr2=0x7f0d880c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74004da0 0x7f0d880c2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 -- 172.21.15.57:0/3629650990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88170f40 con 0x7f0d880aeff0 2023-10-07T23:21:50.811 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8c882700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800d7e0 tx=0x7f0d78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78007360 con 0x7f0d880aeff0 2023-10-07T23:21:50.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d780074c0 con 0x7f0d880aeff0 2023-10-07T23:21:50.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78004830 con 0x7f0d880aeff0 2023-10-07T23:21:50.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d88171220 con 0x7f0d880aeff0 2023-10-07T23:21:50.812 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.802+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d880ac1d0 con 0x7f0d880aeff0 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880711c0 con 0x7f0d880aeff0 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0d78004ba0 con 0x7f0d880aeff0 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d7d7fa700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 0x7f0d6c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d7807c220 con 0x7f0d880aeff0 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d8d083700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 0x7f0d6c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:50.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.806+0000 7f0d8d083700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 0x7f0d6c062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006fd0 tx=0x7f0d84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:50.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.814+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d7804c8c0 con 0x7f0d880aeff0 2023-10-07T23:21:50.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.974+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d880ab0a0 con 0x7f0d880aeff0 2023-10-07T23:21:50.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.974+0000 7f0d7d7fa700 1 -- 172.21.15.57:0/3629650990 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d7804fee0 con 0x7f0d880aeff0 2023-10-07T23:21:50.980 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 msgr2=0x7f0d6c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 0x7f0d6c062a00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0d84006fd0 tx=0x7f0d84008040 comp rx=0 tx=0).stop 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 msgr2=0x7f0d8816d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800d7e0 tx=0x7f0d78009040 comp rx=0 tx=0).stop 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 shutdown_connections 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d8816dd10 0x7f0d880abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d880aeff0 0x7f0d8816d7d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d6c060550 0x7f0d6c062a00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 --2- 172.21.15.57:0/3629650990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d74004da0 0x7f0d880c2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 >> 172.21.15.57:0/3629650990 conn(0x7f0d880670e0 msgr2=0x7f0d88157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 shutdown_connections 2023-10-07T23:21:50.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:50.978+0000 7f0d8e085700 1 -- 172.21.15.57:0/3629650990 wait complete. 2023-10-07T23:21:50.994 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:51.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 Processor -- start 2023-10-07T23:21:51.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 -- start start 2023-10-07T23:21:51.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5804161a70 0x7f580406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f580415d2a0 0x7f58040c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580406bbf0 con 0x7f580415ced0 2023-10-07T23:21:51.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58040c9010 con 0x7f5804161a70 2023-10-07T23:21:51.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580359e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580406b020 con 0x7f580415d2a0 2023-10-07T23:21:51.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5804161a70 0x7f580406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5804161a70 0x7f580406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50646/0 (socket says 172.21.15.57:50646) 2023-10-07T23:21:51.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 -- 172.21.15.57:0/3760408083 learned_addr learned my addr 172.21.15.57:0/3760408083 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5802d9d700 1 --2- 172.21.15.57:0/3760408083 >> v2:172.21.15.154:3300/0 conn(0x7f580415d2a0 0x7f58040c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 -- 172.21.15.57:0/3760408083 >> v2:172.21.15.154:3300/0 conn(0x7f580415d2a0 msgr2=0x7f58040c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 --2- 172.21.15.57:0/3760408083 >> v2:172.21.15.154:3300/0 conn(0x7f580415d2a0 0x7f58040c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 -- 172.21.15.57:0/3760408083 >> v1:172.21.15.154:6789/0 conn(0x7f580415ced0 legacy=0x7f58040c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580159a700 1 -- 172.21.15.57:0/3760408083 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568711691 0 0) 0x7f580406bbf0 con 0x7f580415ced0 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 -- 172.21.15.57:0/3760408083 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580406b2c0 con 0x7f5804161a70 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f5801d9b700 1 --2- 172.21.15.57:0/3760408083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580406c080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58040cb0e0 tx=0x7f57fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e995b5c2214c91d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580159a700 1 -- 172.21.15.57:0/3760408083 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00e070 con 0x7f5804161a70 2023-10-07T23:21:51.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580159a700 1 -- 172.21.15.57:0/3760408083 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc00ba20 con 0x7f5804161a70 2023-10-07T23:21:51.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.098+0000 7f580159a700 1 -- 172.21.15.57:0/3760408083 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0092e0 con 0x7f5804161a70 2023-10-07T23:21:51.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- 172.21.15.57:0/3760408083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 msgr2=0x7f580406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- 172.21.15.57:0/3760408083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580406c080 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58040cb0e0 tx=0x7f57fc009840 comp rx=0 tx=0).stop 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- 172.21.15.57:0/3760408083 shutdown_connections 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- 172.21.15.57:0/3760408083 >> v2:172.21.15.154:3300/0 conn(0x7f580415d2a0 0x7f58040c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- 172.21.15.57:0/3760408083 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580406c080 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- 172.21.15.57:0/3760408083 >> 172.21.15.57:0/3760408083 conn(0x7f58040670e0 msgr2=0x7f58040674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- 172.21.15.57:0/3760408083 shutdown_connections 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- 172.21.15.57:0/3760408083 wait complete. 2023-10-07T23:21:51.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 Processor -- start 2023-10-07T23:21:51.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- start start 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 0x7f58040b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 0x7f58041583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58040b12f0 con 0x7f5804161a70 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58040b0920 con 0x7f580415d2a0 2023-10-07T23:21:51.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.102+0000 7f580359e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58040afd10 con 0x7f580415ced0 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5801d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 0x7f58041583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5801d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 0x7f58041583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:55530/0 (socket says 172.21.15.57:55530) 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5801d9b700 1 -- 172.21.15.57:0/1827046877 learned_addr learned my addr 172.21.15.57:0/1827046877 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f580259c700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 0x7f58040b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 -- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 msgr2=0x7f58040b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 0x7f58040b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 -- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 msgr2=0x7f58041583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 0x7f58041583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 -- 172.21.15.57:0/1827046877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc009ae0 con 0x7f5804161a70 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f580259c700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 0x7f58040b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f5802d9d700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580415e1f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e950 tx=0x7f57f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400c6a0 con 0x7f5804161a70 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f400c820 con 0x7f5804161a70 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58040afff0 con 0x7f5804161a70 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400cbe0 con 0x7f5804161a70 2023-10-07T23:21:51.112 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58040b05b0 con 0x7f5804161a70 2023-10-07T23:21:51.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58040b0a90 con 0x7f5804161a70 2023-10-07T23:21:51.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.106+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f57f4013470 con 0x7f5804161a70 2023-10-07T23:21:51.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.110+0000 7f57fa7fc700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 0x7f57e4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.110+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57f407c650 con 0x7f5804161a70 2023-10-07T23:21:51.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.110+0000 7f580259c700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 0x7f57e4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.115 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.110+0000 7f580259c700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 0x7f57e4062a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57ec007fd0 tx=0x7f57ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.114+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f57f40506d0 con 0x7f5804161a70 2023-10-07T23:21:51.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.278+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58040711c0 con 0x7f5804161a70 2023-10-07T23:21:51.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.278+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1827046877 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f57f4050260 con 0x7f5804161a70 2023-10-07T23:21:51.285 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:51.285 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:51.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 msgr2=0x7f57e4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 0x7f57e4062a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57ec007fd0 tx=0x7f57ec009040 comp rx=0 tx=0).stop 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 msgr2=0x7f580415e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580415e1f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e950 tx=0x7f57f40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 shutdown_connections 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5804161a70 0x7f580415e1f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e4060580 0x7f57e4062a30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f580415d2a0 0x7f58041583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 --2- 172.21.15.57:0/1827046877 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f580415ced0 0x7f58040b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 >> 172.21.15.57:0/1827046877 conn(0x7f58040670e0 msgr2=0x7f58041598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 shutdown_connections 2023-10-07T23:21:51.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.282+0000 7f580359e700 1 -- 172.21.15.57:0/1827046877 wait complete. 2023-10-07T23:21:51.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:51.300 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:51.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.406+0000 7f5102b4f700 1 Processor -- start 2023-10-07T23:21:51.413 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.406+0000 7f5102b4f700 1 -- start start 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50fc0c87d0 0x7f50fc0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c6d20 0x7f50fc0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc0c6890 con 0x7f50fc0c8400 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc0c5ff0 con 0x7f50fc0c87d0 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0c8c00 con 0x7f50fc0c6d20 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50fc0c87d0 0x7f50fc0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50fc0c87d0 0x7f50fc0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50672/0 (socket says 172.21.15.57:50672) 2023-10-07T23:21:51.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 -- 172.21.15.57:0/2550281692 learned_addr learned my addr 172.21.15.57:0/2550281692 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5101b4d700 1 --1- 172.21.15.57:0/2550281692 >> v1:172.21.15.57:6789/0 conn(0x7f50fc0c8400 0x7f50fc0acb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48278/0 (socket says 172.21.15.57:48278) 2023-10-07T23:21:51.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510234e700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c6d20 0x7f50fc0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 -- 172.21.15.57:0/2550281692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c6d20 msgr2=0x7f50fc0c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c6d20 0x7f50fc0c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 -- 172.21.15.57:0/2550281692 >> v1:172.21.15.57:6789/0 conn(0x7f50fc0c8400 legacy=0x7f50fc0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5100b4b700 1 -- 172.21.15.57:0/2550281692 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200702428 0 0) 0x7f50fc0c6890 con 0x7f50fc0c8400 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 -- 172.21.15.57:0/2550281692 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc0ad660 con 0x7f50fc0c87d0 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f510134c700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0ae860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00bc50 tx=0x7f50ec00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60b2ba0569085d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5100b4b700 1 -- 172.21.15.57:0/2550281692 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00dbf0 con 0x7f50fc0c87d0 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5100b4b700 1 -- 172.21.15.57:0/2550281692 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec00dd50 con 0x7f50fc0c87d0 2023-10-07T23:21:51.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5100b4b700 1 -- 172.21.15.57:0/2550281692 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0196b0 con 0x7f50fc0c87d0 2023-10-07T23:21:51.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- 172.21.15.57:0/2550281692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c87d0 msgr2=0x7f50fc0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0ae860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00bc50 tx=0x7f50ec00b4f0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- 172.21.15.57:0/2550281692 shutdown_connections 2023-10-07T23:21:51.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c6d20 0x7f50fc0c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 --2- 172.21.15.57:0/2550281692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0ae860 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- 172.21.15.57:0/2550281692 >> 172.21.15.57:0/2550281692 conn(0x7f50fc0670e0 msgr2=0x7f50fc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- 172.21.15.57:0/2550281692 shutdown_connections 2023-10-07T23:21:51.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.410+0000 7f5102b4f700 1 -- 172.21.15.57:0/2550281692 wait complete. 2023-10-07T23:21:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 Processor -- start 2023-10-07T23:21:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 -- start start 2023-10-07T23:21:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c8400 0x7f50fc14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0aa0f0 con 0x7f50fc0c6d20 2023-10-07T23:21:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0c3f20 con 0x7f50fc0c87d0 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5102b4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0c4200 con 0x7f50fc0c8400 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5101b4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5101b4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50674/0 (socket says 172.21.15.57:50674) 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f5101b4d700 1 -- 172.21.15.57:0/510402451 learned_addr learned my addr 172.21.15.57:0/510402451 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f510234e700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.414+0000 7f510134c700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c8400 0x7f50fc14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 -- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c8400 msgr2=0x7f50fc14eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c8400 0x7f50fc14eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 -- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50fc0c87d0 msgr2=0x7f50fc0b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 -- 172.21.15.57:0/510402451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec00b790 con 0x7f50fc0c6d20 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5101b4d700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c810 tx=0x7f50f8007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8005b40 con 0x7f50fc0c6d20 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8005ca0 con 0x7f50fc0c6d20 2023-10-07T23:21:51.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc0a9aa0 con 0x7f50fc0c6d20 2023-10-07T23:21:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8007320 con 0x7f50fc0c6d20 2023-10-07T23:21:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc14f410 con 0x7f50fc0c6d20 2023-10-07T23:21:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc0711c0 con 0x7f50fc0c6d20 2023-10-07T23:21:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.418+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f50f8012400 con 0x7f50fc0c6d20 2023-10-07T23:21:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.422+0000 7f50f1ffb700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.422+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50f807bd30 con 0x7f50fc0c6d20 2023-10-07T23:21:51.426 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.422+0000 7f510134c700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.422+0000 7f510134c700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00bc50 tx=0x7f50ec01b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.434 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.430+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f50f804c290 con 0x7f50fc0c6d20 2023-10-07T23:21:51.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.586+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50fc0c6240 con 0x7f50fc0c6d20 2023-10-07T23:21:51.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.586+0000 7f50f1ffb700 1 -- 172.21.15.57:0/510402451 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50f80170a0 con 0x7f50fc0c6d20 2023-10-07T23:21:51.592 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 msgr2=0x7f50e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00bc50 tx=0x7f50ec01b000 comp rx=0 tx=0).stop 2023-10-07T23:21:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 msgr2=0x7f50fc0bf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c810 tx=0x7f50f8007de0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 shutdown_connections 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50fc0c87d0 0x7f50fc0b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50fc0c8400 0x7f50fc14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 --2- 172.21.15.57:0/510402451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50fc0c6d20 0x7f50fc0bf4b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 >> 172.21.15.57:0/510402451 conn(0x7f50fc0670e0 msgr2=0x7f50fc06a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 shutdown_connections 2023-10-07T23:21:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.590+0000 7f5102b4f700 1 -- 172.21.15.57:0/510402451 wait complete. 2023-10-07T23:21:51.607 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:51.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 Processor -- start 2023-10-07T23:21:51.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 -- start start 2023-10-07T23:21:51.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f99c8159030 0x7f99c80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 0x7f99c806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c80ca600 0x7f99c80caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c80b0450 con 0x7f99c8159030 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c806a900 con 0x7f99c806c800 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99d02d3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c806a600 con 0x7f99c80ca600 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99ce06f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f99c8159030 0x7f99c80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99ce06f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f99c8159030 0x7f99c80aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50690/0 (socket says 172.21.15.57:50690) 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99ce06f700 1 -- 172.21.15.57:0/2167284849 learned_addr learned my addr 172.21.15.57:0/2167284849 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99cd86e700 1 --2- 172.21.15.57:0/2167284849 >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 0x7f99c806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.718+0000 7f99ce870700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c80ca600 0x7f99c80caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 -- 172.21.15.57:0/2167284849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c80ca600 msgr2=0x7f99c80caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:51.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c80ca600 0x7f99c80caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 -- 172.21.15.57:0/2167284849 >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 msgr2=0x7f99c806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:51.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 --2- 172.21.15.57:0/2167284849 >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 0x7f99c806cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 -- 172.21.15.57:0/2167284849 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c806d170 con 0x7f99c8159030 2023-10-07T23:21:51.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99cd86e700 1 --2- 172.21.15.57:0/2167284849 >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 0x7f99c806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:51.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99ce06f700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c8159030 0x7f99c80aff10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99c80ca540 tx=0x7f99bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c6dcb4646e9e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99cd06d700 1 -- 172.21.15.57:0/2167284849 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00b860 con 0x7f99c8159030 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99cd06d700 1 -- 172.21.15.57:0/2167284849 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00b9c0 con 0x7f99c8159030 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99cd06d700 1 -- 172.21.15.57:0/2167284849 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00bd60 con 0x7f99c8159030 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- 172.21.15.57:0/2167284849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c8159030 msgr2=0x7f99c80aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c8159030 0x7f99c80aff10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99c80ca540 tx=0x7f99bc009250 comp rx=0 tx=0).stop 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- 172.21.15.57:0/2167284849 shutdown_connections 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c80ca600 0x7f99c80caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 --2- 172.21.15.57:0/2167284849 >> v2:172.21.15.154:3300/0 conn(0x7f99c806c800 0x7f99c806cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 --2- 172.21.15.57:0/2167284849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c8159030 0x7f99c80aff10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- 172.21.15.57:0/2167284849 >> 172.21.15.57:0/2167284849 conn(0x7f99c8068650 msgr2=0x7f99c8068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- 172.21.15.57:0/2167284849 shutdown_connections 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- 172.21.15.57:0/2167284849 wait complete. 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 Processor -- start 2023-10-07T23:21:51.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.722+0000 7f99d02d3700 1 -- start start 2023-10-07T23:21:51.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c806c800 0x7f99c8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c8159030 0x7f99c815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80c68b0 con 0x7f99c80ca600 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c815be10 con 0x7f99c8159030 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c80c62a0 con 0x7f99c806c800 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50692/0 (socket says 172.21.15.57:50692) 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 -- 172.21.15.57:0/703991701 learned_addr learned my addr 172.21.15.57:0/703991701 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99ce870700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c8159030 0x7f99c815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99ce06f700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c806c800 0x7f99c8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 -- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c806c800 msgr2=0x7f99c8149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c806c800 0x7f99c8149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 -- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c8159030 msgr2=0x7f99c815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c8159030 0x7f99c815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 -- 172.21.15.57:0/703991701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0094f0 con 0x7f99c80ca600 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99cd86e700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e900 tx=0x7f99c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4013410 con 0x7f99c80ca600 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c4013570 con 0x7f99c80ca600 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c80b1890 con 0x7f99c80ca600 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4013910 con 0x7f99c80ca600 2023-10-07T23:21:51.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c80b1b70 con 0x7f99c80ca600 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.726+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c80647b0 con 0x7f99c80ca600 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.730+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f99c4013cc0 con 0x7f99c80ca600 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.730+0000 7f99ba7fc700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 0x7f99ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.730+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f99c407c420 con 0x7f99c80ca600 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.730+0000 7f99ce06f700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 0x7f99ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:51.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.730+0000 7f99ce06f700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 0x7f99ac062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99c80ca540 tx=0x7f99bc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:51.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.738+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f99c40504a0 con 0x7f99c80ca600 2023-10-07T23:21:51.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99c80cb350 con 0x7f99c80ca600 2023-10-07T23:21:51.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99ba7fc700 1 -- 172.21.15.57:0/703991701 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99c4050030 con 0x7f99c80ca600 2023-10-07T23:21:51.907 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 msgr2=0x7f99ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 0x7f99ac062a00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99c80ca540 tx=0x7f99bc017000 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 msgr2=0x7f99c80b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99c400e900 tx=0x7f99c4008040 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 shutdown_connections 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f99ac060550 0x7f99ac062a00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c8159030 0x7f99c815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99c80ca600 0x7f99c80b20a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 --2- 172.21.15.57:0/703991701 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f99c806c800 0x7f99c8149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:51.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 >> 172.21.15.57:0/703991701 conn(0x7f99c8068650 msgr2=0x7f99c815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:51.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 shutdown_connections 2023-10-07T23:21:51.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:51.902+0000 7f99d02d3700 1 -- 172.21.15.57:0/703991701 wait complete. 2023-10-07T23:21:51.920 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:52.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 Processor -- start 2023-10-07T23:21:52.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 -- start start 2023-10-07T23:21:52.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b80c7ed0 con 0x7f64b806c800 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b806a130 con 0x7f64b80caf20 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64bd351700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80cb770 con 0x7f64b80af9a0 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56044/0 (socket says 172.21.15.57:56044) 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:55558/0 (socket says 172.21.15.57:55558) 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 -- 172.21.15.57:0/2412946454 learned_addr learned my addr 172.21.15.57:0/2412946454 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 -- 172.21.15.57:0/2412946454 >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 msgr2=0x7f64b80c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 --2- 172.21.15.57:0/2412946454 >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.030+0000 7f64b67fc700 1 -- 172.21.15.57:0/2412946454 >> v1:172.21.15.154:6789/0 conn(0x7f64b806c800 legacy=0x7f64b806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b67fc700 1 -- 172.21.15.57:0/2412946454 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80cba50 con 0x7f64b80af9a0 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b77fe700 1 --2- 172.21.15.57:0/2412946454 >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b67fc700 1 --2- 172.21.15.57:0/2412946454 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0099b0 tx=0x7f64ac009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eadab9b950471d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b5ffb700 1 -- 172.21.15.57:0/2412946454 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac009ea0 con 0x7f64b80af9a0 2023-10-07T23:21:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b5ffb700 1 -- 172.21.15.57:0/2412946454 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac00b860 con 0x7f64b80af9a0 2023-10-07T23:21:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64b5ffb700 1 -- 172.21.15.57:0/2412946454 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00bc00 con 0x7f64b80af9a0 2023-10-07T23:21:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 -- 172.21.15.57:0/2412946454 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 msgr2=0x7f64b8069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 --2- 172.21.15.57:0/2412946454 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0099b0 tx=0x7f64ac009250 comp rx=0 tx=0).stop 2023-10-07T23:21:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 -- 172.21.15.57:0/2412946454 shutdown_connections 2023-10-07T23:21:52.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 --2- 172.21.15.57:0/2412946454 >> v2:172.21.15.154:3300/0 conn(0x7f64b80caf20 0x7f64b80c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 --2- 172.21.15.57:0/2412946454 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b80af9a0 0x7f64b8069bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 -- 172.21.15.57:0/2412946454 >> 172.21.15.57:0/2412946454 conn(0x7f64b806e410 msgr2=0x7f64b8067710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 -- 172.21.15.57:0/2412946454 shutdown_connections 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.034+0000 7f64bd351700 1 -- 172.21.15.57:0/2412946454 wait complete. 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 Processor -- start 2023-10-07T23:21:52.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- start start 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b806c800 0x7f64b8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b80af9a0 0x7f64b80c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b81561a0 con 0x7f64b80caf20 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8159ef0 con 0x7f64b80af9a0 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b815a140 con 0x7f64b806c800 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50704/0 (socket says 172.21.15.57:50704) 2023-10-07T23:21:52.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 -- 172.21.15.57:0/3226324411 learned_addr learned my addr 172.21.15.57:0/3226324411 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b6ffd700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b806c800 0x7f64b8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b67fc700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b80af9a0 0x7f64b80c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 -- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b806c800 msgr2=0x7f64b8155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b806c800 0x7f64b8155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 -- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b80af9a0 msgr2=0x7f64b80c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b80af9a0 0x7f64b80c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 -- 172.21.15.57:0/3226324411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b0009a20 con 0x7f64b80caf20 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64b77fe700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f20 tx=0x7f64b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b000cb70 con 0x7f64b80caf20 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b000f040 con 0x7f64b80caf20 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac0094f0 con 0x7f64b80caf20 2023-10-07T23:21:52.046 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0014480 con 0x7f64b80caf20 2023-10-07T23:21:52.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.038+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b815ce40 con 0x7f64b80caf20 2023-10-07T23:21:52.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b815a530 con 0x7f64b80caf20 2023-10-07T23:21:52.047 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f64b0003020 con 0x7f64b80caf20 2023-10-07T23:21:52.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f649effd700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 0x7f64a4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64b007d0a0 con 0x7f64b80caf20 2023-10-07T23:21:52.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f64b6ffd700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 0x7f64a4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.042+0000 7f64b6ffd700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 0x7f64a4062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64a80039f0 tx=0x7f64a8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.050+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f64b004d5f0 con 0x7f64b80caf20 2023-10-07T23:21:52.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.210+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64b815a300 con 0x7f64b80caf20 2023-10-07T23:21:52.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.210+0000 7f649effd700 1 -- 172.21.15.57:0/3226324411 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64b0050c10 con 0x7f64b80caf20 2023-10-07T23:21:52.217 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:52.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 msgr2=0x7f64a4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 0x7f64a4062a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64a80039f0 tx=0x7f64a8007ec0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 msgr2=0x7f64b815aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009f20 tx=0x7f64b000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 shutdown_connections 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64b80caf20 0x7f64b815aa70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f64a4060550 0x7f64a4062a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b80af9a0 0x7f64b80c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 --2- 172.21.15.57:0/3226324411 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f64b806c800 0x7f64b8155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 >> 172.21.15.57:0/3226324411 conn(0x7f64b806e410 msgr2=0x7f64b81589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 shutdown_connections 2023-10-07T23:21:52.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.214+0000 7f64bd351700 1 -- 172.21.15.57:0/3226324411 wait complete. 2023-10-07T23:21:52.231 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 Processor -- start 2023-10-07T23:21:52.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- start start 2023-10-07T23:21:52.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 0x7f62b4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62b415a210 0x7f62b40c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b40ca6c0 con 0x7f62b40cb6d0 2023-10-07T23:21:52.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b40cb240 con 0x7f62b415a210 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b40ca910 con 0x7f62b4155780 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b27fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f62b40cb6d0 0x7f62b4069870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48282/0 (socket says 172.21.15.57:48282) 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b27fc700 1 -- 172.21.15.57:0/2842922182 learned_addr learned my addr 172.21.15.57:0/2842922182 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 --2- 172.21.15.57:0/2842922182 >> v2:172.21.15.154:3300/0 conn(0x7f62b415a210 0x7f62b40c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b2ffd700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 0x7f62b4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095709081 0 0) 0x7f62b40ca6c0 con 0x7f62b40cb6d0 2023-10-07T23:21:52.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c003570 con 0x7f62b40cb6d0 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 -- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 msgr2=0x7f62b4155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 0x7f62b4155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634100910 0 0) 0x7f629c003570 con 0x7f62b40cb6d0 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 -- 172.21.15.57:0/2842922182 >> v1:172.21.15.57:6789/0 conn(0x7f62b40cb6d0 legacy=0x7f62b4069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 -- 172.21.15.57:0/2842922182 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40cabf0 con 0x7f62b415a210 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b2ffd700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 0x7f62b4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b37fe700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b415a210 0x7f62b40c8670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009e60 tx=0x7f62ac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4acd312e1f7c149f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00e070 con 0x7f62b415a210 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac00b7a0 con 0x7f62b415a210 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b1ffb700 1 -- 172.21.15.57:0/2842922182 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4001470 con 0x7f62b415a210 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b415a210 msgr2=0x7f62b40c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b415a210 0x7f62b40c8670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009e60 tx=0x7f62ac009780 comp rx=0 tx=0).stop 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- 172.21.15.57:0/2842922182 shutdown_connections 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b415a210 0x7f62b40c8670 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 --2- 172.21.15.57:0/2842922182 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b4155780 0x7f62b4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.334+0000 7f62b3fff700 1 -- 172.21.15.57:0/2842922182 >> 172.21.15.57:0/2842922182 conn(0x7f62b406b5d0 msgr2=0x7f62b406b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- 172.21.15.57:0/2842922182 shutdown_connections 2023-10-07T23:21:52.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- 172.21.15.57:0/2842922182 wait complete. 2023-10-07T23:21:52.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 Processor -- start 2023-10-07T23:21:52.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- start start 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 0x7f62b4153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b415a210 0x7f62b4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4157ea0 con 0x7f62b40cb6d0 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4158140 con 0x7f62b4155780 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b3fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4158420 con 0x7f62b415a210 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50716/0 (socket says 172.21.15.57:50716) 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 -- 172.21.15.57:0/3805093977 learned_addr learned my addr 172.21.15.57:0/3805093977 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b37fe700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b415a210 0x7f62b4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b27fc700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 0x7f62b4153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 -- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b415a210 msgr2=0x7f62b4155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b415a210 0x7f62b4155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 -- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 msgr2=0x7f62b4153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 0x7f62b4153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b2ffd700 1 -- 172.21.15.57:0/3805093977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80093c0 con 0x7f62b40cb6d0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.338+0000 7f62b27fc700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 0x7f62b4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f62b2ffd700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62a800ec80 tx=0x7f62a8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8005a00 con 0x7f62b40cb6d0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a8005b60 con 0x7f62b40cb6d0 2023-10-07T23:21:52.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac009a20 con 0x7f62b40cb6d0 2023-10-07T23:21:52.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a800c920 con 0x7f62b40cb6d0 2023-10-07T23:21:52.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b415d630 con 0x7f62b40cb6d0 2023-10-07T23:21:52.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f62a800cc90 con 0x7f62b40cb6d0 2023-10-07T23:21:52.348 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b40711c0 con 0x7f62b40cb6d0 2023-10-07T23:21:52.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 0x7f62a0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62a807cd30 con 0x7f62b40cb6d0 2023-10-07T23:21:52.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.342+0000 7f62b27fc700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 0x7f62a0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.346+0000 7f62b27fc700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 0x7f62a0062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62b40cbe40 tx=0x7f62a4001580 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.350+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62a804d320 con 0x7f62b40cb6d0 2023-10-07T23:21:52.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.510+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62b40af890 con 0x7f62b40cb6d0 2023-10-07T23:21:52.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.514+0000 7f629affd700 1 -- 172.21.15.57:0/3805093977 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f62a8020090 con 0x7f62b40cb6d0 2023-10-07T23:21:52.520 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:52.520 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.514+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 msgr2=0x7f62a0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.514+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 0x7f62a0062a00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62b40cbe40 tx=0x7f62a4001580 comp rx=0 tx=0).stop 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 msgr2=0x7f62b415d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62a800ec80 tx=0x7f62a8009120 comp rx=0 tx=0).stop 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 shutdown_connections 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62b415a210 0x7f62b4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62a0060550 0x7f62a0062a00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62b4155780 0x7f62b4153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 --2- 172.21.15.57:0/3805093977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62b40cb6d0 0x7f62b415d0f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 >> 172.21.15.57:0/3805093977 conn(0x7f62b406b5d0 msgr2=0x7f62b41518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 shutdown_connections 2023-10-07T23:21:52.523 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.518+0000 7f62b3fff700 1 -- 172.21.15.57:0/3805093977 wait complete. 2023-10-07T23:21:52.538 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:52.538 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:52.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 Processor -- start 2023-10-07T23:21:52.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 -- start start 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5180c97e0 0x7fc5180ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5180acd40 0x7fc518068300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc518157ed0 con 0x7fc518156e30 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc518158460 con 0x7fc5180c97e0 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51d1a7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc518158180 con 0x7fc5180acd40 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5180c97e0 0x7fc5180ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc5180c97e0 0x7fc5180ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50724/0 (socket says 172.21.15.57:50724) 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51659c700 1 -- 172.21.15.57:0/2765043993 learned_addr learned my addr 172.21.15.57:0/2765043993 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc51759e700 1 --2- 172.21.15.57:0/2765043993 >> v2:172.21.15.154:3300/0 conn(0x7fc5180acd40 0x7fc518068300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.646+0000 7fc515d9b700 1 -- 172.21.15.57:0/2765043993 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112309483 0 0) 0x7fc518157ed0 con 0x7fc518156e30 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc515d9b700 1 -- 172.21.15.57:0/2765043993 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500003570 con 0x7fc518156e30 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51659c700 1 -- 172.21.15.57:0/2765043993 >> v2:172.21.15.154:3300/0 conn(0x7fc5180acd40 msgr2=0x7fc518068300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51659c700 1 --2- 172.21.15.57:0/2765043993 >> v2:172.21.15.154:3300/0 conn(0x7fc5180acd40 0x7fc518068300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51659c700 1 -- 172.21.15.57:0/2765043993 >> v1:172.21.15.154:6789/0 conn(0x7fc518156e30 legacy=0x7fc5180ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51659c700 1 -- 172.21.15.57:0/2765043993 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180ac150 con 0x7fc5180c97e0 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51659c700 1 --2- 172.21.15.57:0/2765043993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5180c97e0 0x7fc5180ac800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc5180c9cc0 tx=0x7fc50c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a035d6f1ddf28d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc515d9b700 1 -- 172.21.15.57:0/2765043993 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00b640 con 0x7fc5180c97e0 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc515d9b700 1 -- 172.21.15.57:0/2765043993 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c00b7a0 con 0x7fc5180c97e0 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc515d9b700 1 -- 172.21.15.57:0/2765043993 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00bb60 con 0x7fc5180c97e0 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 -- 172.21.15.57:0/2765043993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5180c97e0 msgr2=0x7fc5180ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/2765043993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5180c97e0 0x7fc5180ac800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc5180c9cc0 tx=0x7fc50c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 -- 172.21.15.57:0/2765043993 shutdown_connections 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/2765043993 >> v2:172.21.15.154:3300/0 conn(0x7fc5180acd40 0x7fc518068300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/2765043993 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc5180c97e0 0x7fc5180ac800 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 -- 172.21.15.57:0/2765043993 >> 172.21.15.57:0/2765043993 conn(0x7fc5180ccf10 msgr2=0x7fc5180c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 -- 172.21.15.57:0/2765043993 shutdown_connections 2023-10-07T23:21:52.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.650+0000 7fc51d1a7700 1 -- 172.21.15.57:0/2765043993 wait complete. 2023-10-07T23:21:52.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 Processor -- start 2023-10-07T23:21:52.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 -- start start 2023-10-07T23:21:52.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5180acd40 0x7fc518148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5180c97e0 0x7fc518151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5181559c0 con 0x7fc518156e30 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc518154920 con 0x7fc5180c97e0 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51d1a7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc518154c00 con 0x7fc5180acd40 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50726/0 (socket says 172.21.15.57:50726) 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 -- 172.21.15.57:0/1879953705 learned_addr learned my addr 172.21.15.57:0/1879953705 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc516d9d700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5180acd40 0x7fc518148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51659c700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5180c97e0 0x7fc518151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 -- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5180acd40 msgr2=0x7fc518148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5180acd40 0x7fc518148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 -- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5180c97e0 msgr2=0x7fc518151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5180c97e0 0x7fc518151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 -- 172.21.15.57:0/1879953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0094f0 con 0x7fc518156e30 2023-10-07T23:21:52.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc51759e700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc510009df0 tx=0x7fc51000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000caa0 con 0x7fc518156e30 2023-10-07T23:21:52.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.654+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000f040 con 0x7fc518156e30 2023-10-07T23:21:52.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc518154ee0 con 0x7fc518156e30 2023-10-07T23:21:52.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000ce70 con 0x7fc518156e30 2023-10-07T23:21:52.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5181554a0 con 0x7fc518156e30 2023-10-07T23:21:52.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518155690 con 0x7fc518156e30 2023-10-07T23:21:52.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc510014690 con 0x7fc518156e30 2023-10-07T23:21:52.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.658+0000 7fc4feffd700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 0x7fc504062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.662+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc51007cfe0 con 0x7fc518156e30 2023-10-07T23:21:52.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.662+0000 7fc516d9d700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 0x7fc504062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.662+0000 7fc516d9d700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 0x7fc504062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.670+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc510051020 con 0x7fc518156e30 2023-10-07T23:21:52.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.830+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5180647b0 con 0x7fc518156e30 2023-10-07T23:21:52.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.830+0000 7fc4feffd700 1 -- 172.21.15.57:0/1879953705 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc510018020 con 0x7fc518156e30 2023-10-07T23:21:52.836 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:52.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 msgr2=0x7fc504062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 0x7fc504062a00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508007d50 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 msgr2=0x7fc5180b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc510009df0 tx=0x7fc51000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 shutdown_connections 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc518156e30 0x7fc5180b0ba0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc504060550 0x7fc504062a00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5180c97e0 0x7fc518151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 --2- 172.21.15.57:0/1879953705 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc5180acd40 0x7fc518148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 >> 172.21.15.57:0/1879953705 conn(0x7fc5180ccf10 msgr2=0x7fc51806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 shutdown_connections 2023-10-07T23:21:52.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.834+0000 7fc51d1a7700 1 -- 172.21.15.57:0/1879953705 wait complete. 2023-10-07T23:21:52.851 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:52.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.954+0000 7f7a63fff700 1 Processor -- start 2023-10-07T23:21:52.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.954+0000 7f7a63fff700 1 -- start start 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a640ca8f0 0x7f7a64158a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a64158f70 0x7f7a640c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6406c090 con 0x7f7a640ca520 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a640c96f0 con 0x7f7a640ca8f0 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6406b480 con 0x7f7a64158f70 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a640ca8f0 0x7f7a64158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7a640ca8f0 0x7f7a64158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50736/0 (socket says 172.21.15.57:50736) 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 -- 172.21.15.57:0/3926623876 learned_addr learned my addr 172.21.15.57:0/3926623876 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a62ffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7a640ca520 0x7f7a6406a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48286/0 (socket says 172.21.15.57:48286) 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a637fe700 1 --2- 172.21.15.57:0/3926623876 >> v2:172.21.15.154:3300/0 conn(0x7f7a64158f70 0x7f7a640c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 -- 172.21.15.57:0/3926623876 >> v2:172.21.15.154:3300/0 conn(0x7f7a64158f70 msgr2=0x7f7a640c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 --2- 172.21.15.57:0/3926623876 >> v2:172.21.15.154:3300/0 conn(0x7f7a64158f70 0x7f7a640c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 -- 172.21.15.57:0/3926623876 >> v1:172.21.15.57:6789/0 conn(0x7f7a640ca520 legacy=0x7f7a6406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 -- 172.21.15.57:0/3926623876 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6406b760 con 0x7f7a640ca8f0 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a627fc700 1 --2- 172.21.15.57:0/3926623876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a64158a30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a64067df0 tx=0x7f7a54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb534b6da5eb94d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a61ffb700 1 -- 172.21.15.57:0/3926623876 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400b860 con 0x7f7a640ca8f0 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a61ffb700 1 -- 172.21.15.57:0/3926623876 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5400b9c0 con 0x7f7a640ca8f0 2023-10-07T23:21:52.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a61ffb700 1 -- 172.21.15.57:0/3926623876 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400bd60 con 0x7f7a640ca8f0 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- 172.21.15.57:0/3926623876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 msgr2=0x7f7a64158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 --2- 172.21.15.57:0/3926623876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a64158a30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7a64067df0 tx=0x7f7a54009250 comp rx=0 tx=0).stop 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- 172.21.15.57:0/3926623876 shutdown_connections 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 --2- 172.21.15.57:0/3926623876 >> v2:172.21.15.154:3300/0 conn(0x7f7a64158f70 0x7f7a640c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 --2- 172.21.15.57:0/3926623876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a64158a30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- 172.21.15.57:0/3926623876 >> 172.21.15.57:0/3926623876 conn(0x7f7a640cc8c0 msgr2=0x7f7a640c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:52.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- 172.21.15.57:0/3926623876 shutdown_connections 2023-10-07T23:21:52.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.958+0000 7f7a63fff700 1 -- 172.21.15.57:0/3926623876 wait complete. 2023-10-07T23:21:52.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 Processor -- start 2023-10-07T23:21:52.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- start start 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a640ca520 0x7f7a640c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a64158f70 0x7f7a64168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6415b500 con 0x7f7a640ca8f0 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6415adc0 con 0x7f7a64158f70 2023-10-07T23:21:52.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640c3e20 con 0x7f7a640ca520 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50744/0 (socket says 172.21.15.57:50744) 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 -- 172.21.15.57:0/1448376672 learned_addr learned my addr 172.21.15.57:0/1448376672 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a637fe700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a64158f70 0x7f7a64168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a62ffd700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a640ca520 0x7f7a640c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 -- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a640ca520 msgr2=0x7f7a640c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a640ca520 0x7f7a640c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 -- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a64158f70 msgr2=0x7f7a64168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a64158f70 0x7f7a64168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:52.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 -- 172.21.15.57:0/1448376672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540094f0 con 0x7f7a640ca8f0 2023-10-07T23:21:52.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a627fc700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a6406f4e0 tx=0x7f7a54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54024440 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a540245a0 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a640c4100 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54024960 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.962+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a64169380 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.966+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a64169510 con 0x7f7a640ca8f0 2023-10-07T23:21:52.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.966+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7a5401a420 con 0x7f7a640ca8f0 2023-10-07T23:21:52.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.966+0000 7f7a4affd700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 0x7f7a50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:52.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.966+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a5407e600 con 0x7f7a640ca8f0 2023-10-07T23:21:52.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.966+0000 7f7a62ffd700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 0x7f7a50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:52.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.970+0000 7f7a62ffd700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 0x7f7a50062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004fd0 tx=0x7f7a58007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:52.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:52.974+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7a5404ebf0 con 0x7f7a640ca8f0 2023-10-07T23:21:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.138+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a640711c0 con 0x7f7a640ca8f0 2023-10-07T23:21:53.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.138+0000 7f7a4affd700 1 -- 172.21.15.57:0/1448376672 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a54052210 con 0x7f7a640ca8f0 2023-10-07T23:21:53.144 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 msgr2=0x7f7a50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 0x7f7a50062a00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7a58004fd0 tx=0x7f7a58007eb0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 msgr2=0x7f7a6415a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a6406f4e0 tx=0x7f7a54017000 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 shutdown_connections 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7a50060550 0x7f7a50062a00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a64158f70 0x7f7a64168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a640ca8f0 0x7f7a6415a880 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 --2- 172.21.15.57:0/1448376672 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7a640ca520 0x7f7a640c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 >> 172.21.15.57:0/1448376672 conn(0x7f7a640cc8c0 msgr2=0x7f7a640b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 shutdown_connections 2023-10-07T23:21:53.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.142+0000 7f7a63fff700 1 -- 172.21.15.57:0/1448376672 wait complete. 2023-10-07T23:21:53.158 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 Processor -- start 2023-10-07T23:21:53.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 -- start start 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6f58158ce0 0x7f6f5815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f5806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5806c800 con 0x7f6f5815d080 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5806cdc0 con 0x7f6f58158ce0 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.266+0000 7f6f57fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580ae850 con 0x7f6f580c9ad0 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6f5815d080 0x7f6f580c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48290/0 (socket says 172.21.15.57:48290) 2023-10-07T23:21:53.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 -- 172.21.15.57:0/667243532 learned_addr learned my addr 172.21.15.57:0/667243532 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f577fe700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f5806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f56ffd700 1 --2- 172.21.15.57:0/667243532 >> v2:172.21.15.154:3300/0 conn(0x7f6f58158ce0 0x7f6f5815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572403645 0 0) 0x7f6f5806c800 con 0x7f6f5815d080 2023-10-07T23:21:53.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f44003570 con 0x7f6f5815d080 2023-10-07T23:21:53.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082895766 0 0) 0x7f6f44003570 con 0x7f6f5815d080 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 msgr2=0x7f6f5806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f5806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 >> v2:172.21.15.154:3300/0 conn(0x7f6f58158ce0 msgr2=0x7f6f5815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 --2- 172.21.15.57:0/667243532 >> v2:172.21.15.154:3300/0 conn(0x7f6f58158ce0 0x7f6f5815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5815d600 con 0x7f6f5815d080 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f6f48001470 con 0x7f6f5815d080 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 >> v1:172.21.15.57:6789/0 conn(0x7f6f5815d080 legacy=0x7f6f580c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f44003a60 0x7f6f44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f44004810 0x7f6f44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5806cdc0 con 0x7f6f44004810 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580ae850 con 0x7f6f44003a60 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4400ba60 con 0x7f6f580c9ad0 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f44004810 0x7f6f44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f577fe700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f56ffd700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f44003a60 0x7f6f44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 -- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 msgr2=0x7f6f440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f580c9ad0 0x7f6f440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 -- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f44003a60 msgr2=0x7f6f44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f44003a60 0x7f6f44003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 -- 172.21.15.57:0/667243532 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44003570 con 0x7f6f44004810 2023-10-07T23:21:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f567fc700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f44004810 0x7f6f44004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c3f0 tx=0x7f6f48001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa144854aaf984 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800cc80 con 0x7f6f44004810 2023-10-07T23:21:53.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800cde0 con 0x7f6f44004810 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f55ffb700 1 -- 172.21.15.57:0/667243532 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48005b70 con 0x7f6f44004810 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 -- 172.21.15.57:0/667243532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f44004810 msgr2=0x7f6f44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 --2- 172.21.15.57:0/667243532 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f44004810 0x7f6f44004c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c3f0 tx=0x7f6f48001e90 comp rx=0 tx=0).stop 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f577fe700 1 -- 172.21.15.57:0/667243532 reap_dead start 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 -- 172.21.15.57:0/667243532 shutdown_connections 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 -- 172.21.15.57:0/667243532 >> 172.21.15.57:0/667243532 conn(0x7f6f58067b40 msgr2=0x7f6f580c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:53.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 -- 172.21.15.57:0/667243532 shutdown_connections 2023-10-07T23:21:53.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.270+0000 7f6f57fff700 1 -- 172.21.15.57:0/667243532 wait complete. 2023-10-07T23:21:53.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.274+0000 7f6f57fff700 1 Processor -- start 2023-10-07T23:21:53.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.274+0000 7f6f57fff700 1 -- start start 2023-10-07T23:21:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f44004810 0x7f6f5814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f58154cd0 0x7f6f58157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58156c60 con 0x7f6f580c84a0 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580c8e80 con 0x7f6f58154cd0 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581585e0 con 0x7f6f44004810 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50758/0 (socket says 172.21.15.57:50758) 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 -- 172.21.15.57:0/3152938338 learned_addr learned my addr 172.21.15.57:0/3152938338 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f567fc700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f58154cd0 0x7f6f58157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f56ffd700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f44004810 0x7f6f5814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 -- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f44004810 msgr2=0x7f6f5814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f44004810 0x7f6f5814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 -- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f58154cd0 msgr2=0x7f6f58157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f58154cd0 0x7f6f58157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 -- 172.21.15.57:0/3152938338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5814f330 con 0x7f6f580c84a0 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f577fe700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f5806f580 tx=0x7f6f5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000e070 con 0x7f6f580c84a0 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f50011b80 con 0x7f6f580c84a0 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f58156e50 con 0x7f6f580c84a0 2023-10-07T23:21:53.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50016470 con 0x7f6f580c84a0 2023-10-07T23:21:53.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.278+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f58157410 con 0x7f6f580c84a0 2023-10-07T23:21:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f580711c0 con 0x7f6f580c84a0 2023-10-07T23:21:53.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6f50016860 con 0x7f6f580c84a0 2023-10-07T23:21:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f4effd700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 0x7f6f38062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f5007d200 con 0x7f6f580c84a0 2023-10-07T23:21:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f56ffd700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 0x7f6f38062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.282+0000 7f6f56ffd700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 0x7f6f38062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ee0 tx=0x7f6f40008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.286+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6f5004d7f0 con 0x7f6f580c84a0 2023-10-07T23:21:53.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.450+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f58157a90 con 0x7f6f580c84a0 2023-10-07T23:21:53.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.450+0000 7f6f4effd700 1 -- 172.21.15.57:0/3152938338 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f50050e10 con 0x7f6f580c84a0 2023-10-07T23:21:53.457 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 msgr2=0x7f6f38062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 0x7f6f38062a00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ee0 tx=0x7f6f40008040 comp rx=0 tx=0).stop 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 msgr2=0x7f6f580c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6f5806f580 tx=0x7f6f5000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 shutdown_connections 2023-10-07T23:21:53.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6f580c84a0 0x7f6f580c8940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6f38060550 0x7f6f38062a00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f58154cd0 0x7f6f58157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 --2- 172.21.15.57:0/3152938338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6f44004810 0x7f6f5814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 >> 172.21.15.57:0/3152938338 conn(0x7f6f58067b40 msgr2=0x7f6f580af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 shutdown_connections 2023-10-07T23:21:53.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.454+0000 7f6f57fff700 1 -- 172.21.15.57:0/3152938338 wait complete. 2023-10-07T23:21:53.480 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2023-10-07T23:21:53.481 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:21:53.481 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-07T23:21:53.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.582+0000 7f439e35c700 1 Processor -- start 2023-10-07T23:21:53.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.582+0000 7f439e35c700 1 -- start start 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f43980ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4398158fb0 0x7f43980ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439806b3c0 con 0x7f439806b850 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439806a7b0 con 0x7f4398158fb0 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439806aa90 con 0x7f4398158be0 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439db5b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f439806b850 0x7f43980cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48298/0 (socket says 172.21.15.57:48298) 2023-10-07T23:21:53.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439db5b700 1 -- 172.21.15.57:0/252316866 learned_addr learned my addr 172.21.15.57:0/252316866 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:53.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 --2- 172.21.15.57:0/252316866 >> v2:172.21.15.154:3300/0 conn(0x7f4398158fb0 0x7f43980ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439d35a700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f43980ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f438bfff700 1 -- 172.21.15.57:0/252316866 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277914021 0 0) 0x7f439806b3c0 con 0x7f439806b850 2023-10-07T23:21:53.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f438bfff700 1 -- 172.21.15.57:0/252316866 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4384003570 con 0x7f439806b850 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 -- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 msgr2=0x7f43980ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f43980ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 -- 172.21.15.57:0/252316866 >> v1:172.21.15.57:6789/0 conn(0x7f439806b850 legacy=0x7f43980cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 -- 172.21.15.57:0/252316866 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439806ad70 con 0x7f4398158fb0 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439cb59700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980ce6b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f438c0098e0 tx=0x7f438c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1f47da60d1f303 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f438bfff700 1 -- 172.21.15.57:0/252316866 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c011630 con 0x7f4398158fb0 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f438bfff700 1 -- 172.21.15.57:0/252316866 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c011790 con 0x7f4398158fb0 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f438bfff700 1 -- 172.21.15.57:0/252316866 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c011b30 con 0x7f4398158fb0 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 msgr2=0x7f43980ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980ce6b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f438c0098e0 tx=0x7f438c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- 172.21.15.57:0/252316866 shutdown_connections 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980ce6b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 --2- 172.21.15.57:0/252316866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f43980ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- 172.21.15.57:0/252316866 >> 172.21.15.57:0/252316866 conn(0x7f4398068520 msgr2=0x7f4398068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- 172.21.15.57:0/252316866 shutdown_connections 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.586+0000 7f439e35c700 1 -- 172.21.15.57:0/252316866 wait complete. 2023-10-07T23:21:53.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 Processor -- start 2023-10-07T23:21:53.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 -- start start 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f439815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43980c6e40 con 0x7f439806b850 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439815bb50 con 0x7f4398158fb0 2023-10-07T23:21:53.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439e35c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43981697b0 con 0x7f4398158be0 2023-10-07T23:21:53.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50770/0 (socket says 172.21.15.57:50770) 2023-10-07T23:21:53.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 -- 172.21.15.57:0/3258833251 learned_addr learned my addr 172.21.15.57:0/3258833251 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:53.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439db5b700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439cb59700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f439815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 -- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 msgr2=0x7f439815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f439815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 -- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 msgr2=0x7f43980c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 -- 172.21.15.57:0/3258833251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c009420 con 0x7f439806b850 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439db5b700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f439d35a700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f439400ec80 tx=0x7f43940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.590+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439400c850 con 0x7f439806b850 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439400c9b0 con 0x7f439806b850 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4398169a90 con 0x7f439806b850 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439400cd50 con 0x7f439806b850 2023-10-07T23:21:53.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439816a050 con 0x7f439806b850 2023-10-07T23:21:53.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43980711c0 con 0x7f439806b850 2023-10-07T23:21:53.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4394013680 con 0x7f439806b850 2023-10-07T23:21:53.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f43897fa700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 0x7f437c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:53.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f439407ca40 con 0x7f439806b850 2023-10-07T23:21:53.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.594+0000 7f439cb59700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 0x7f437c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:53.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.598+0000 7f439cb59700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 0x7f437c062a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f438c0091e0 tx=0x7f438c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:53.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.606+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f439404d010 con 0x7f439806b850 2023-10-07T23:21:53.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.830+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7f439806d4c0 con 0x7f439806b850 2023-10-07T23:21:53.873 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f43897fa700 1 -- 172.21.15.57:0/3258833251 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v51) v1 ==== 168+0+0 (secure 0 0 0) 0x7f4394050630 con 0x7f439806b850 2023-10-07T23:21:53.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 msgr2=0x7f437c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 0x7f437c062a30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f438c0091e0 tx=0x7f438c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 msgr2=0x7f4398169180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f439400ec80 tx=0x7f43940091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 shutdown_connections 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f437c060580 0x7f437c062a30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4398158fb0 0x7f43980c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4398158be0 0x7f439815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 --2- 172.21.15.57:0/3258833251 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f439806b850 0x7f4398169180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:53.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.870+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 >> 172.21.15.57:0/3258833251 conn(0x7f4398068520 msgr2=0x7f43980ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:53.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.874+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 shutdown_connections 2023-10-07T23:21:53.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.874+0000 7f439e35c700 1 -- 172.21.15.57:0/3258833251 wait complete. 2023-10-07T23:21:53.894 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-07T23:21:54.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 Processor -- start 2023-10-07T23:21:54.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 -- start start 2023-10-07T23:21:54.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5604158ce0 0x7f560415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f56040c9ad0 0x7f560406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560406c800 con 0x7f560415d080 2023-10-07T23:21:54.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560406cdc0 con 0x7f56040c9ad0 2023-10-07T23:21:54.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560abea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56040ae850 con 0x7f5604158ce0 2023-10-07T23:21:54.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:53.998+0000 7f560a3e9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f56040c9ad0 0x7f560406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f56093e7700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f560415d080 0x7f56040c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:56602/0 (socket says 172.21.15.57:56602) 2023-10-07T23:21:54.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f56093e7700 1 -- 172.21.15.57:0/845185357 learned_addr learned my addr 172.21.15.57:0/845185357 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:54.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5609be8700 1 --2- 172.21.15.57:0/845185357 >> v2:172.21.15.154:3300/0 conn(0x7f5604158ce0 0x7f560415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5608be6700 1 -- 172.21.15.57:0/845185357 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216830433 0 0) 0x7f560406c800 con 0x7f560415d080 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5608be6700 1 -- 172.21.15.57:0/845185357 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f0003570 con 0x7f560415d080 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560a3e9700 1 -- 172.21.15.57:0/845185357 >> v2:172.21.15.154:3300/0 conn(0x7f5604158ce0 msgr2=0x7f560415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560a3e9700 1 --2- 172.21.15.57:0/845185357 >> v2:172.21.15.154:3300/0 conn(0x7f5604158ce0 0x7f560415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560a3e9700 1 -- 172.21.15.57:0/845185357 >> v1:172.21.15.154:6789/0 conn(0x7f560415d080 legacy=0x7f56040c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560a3e9700 1 -- 172.21.15.57:0/845185357 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560415d600 con 0x7f56040c9ad0 2023-10-07T23:21:54.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560a3e9700 1 --2- 172.21.15.57:0/845185357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f560406d250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f56040c9f00 tx=0x7f55fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636e5b7136b2efe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5608be6700 1 -- 172.21.15.57:0/845185357 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00e070 con 0x7f56040c9ad0 2023-10-07T23:21:54.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5608be6700 1 -- 172.21.15.57:0/845185357 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00bab0 con 0x7f56040c9ad0 2023-10-07T23:21:54.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f5608be6700 1 -- 172.21.15.57:0/845185357 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00be30 con 0x7f56040c9ad0 2023-10-07T23:21:54.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 -- 172.21.15.57:0/845185357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 msgr2=0x7f560406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 --2- 172.21.15.57:0/845185357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f560406d250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f56040c9f00 tx=0x7f55fc009780 comp rx=0 tx=0).stop 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 -- 172.21.15.57:0/845185357 shutdown_connections 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 --2- 172.21.15.57:0/845185357 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f560406d250 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 --2- 172.21.15.57:0/845185357 >> v2:172.21.15.154:3300/0 conn(0x7f5604158ce0 0x7f560415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 -- 172.21.15.57:0/845185357 >> 172.21.15.57:0/845185357 conn(0x7f5604067b40 msgr2=0x7f56040c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 -- 172.21.15.57:0/845185357 shutdown_connections 2023-10-07T23:21:54.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.002+0000 7f560abea700 1 -- 172.21.15.57:0/845185357 wait complete. 2023-10-07T23:21:54.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 Processor -- start 2023-10-07T23:21:54.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- start start 2023-10-07T23:21:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5604158ce0 0x7f56040b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 0x7f56040b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040a90f0 con 0x7f56040c9ad0 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040afed0 con 0x7f5604158ce0 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56040b01b0 con 0x7f560415d080 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50796/0 (socket says 172.21.15.57:50796) 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 -- 172.21.15.57:0/1554119347 learned_addr learned my addr 172.21.15.57:0/1554119347 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f56093e7700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5604158ce0 0x7f56040b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560a3e9700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 0x7f56040b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 -- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 msgr2=0x7f56040b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 0x7f56040b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 -- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5604158ce0 msgr2=0x7f56040b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5604158ce0 0x7f56040b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 -- 172.21.15.57:0/1554119347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc009a20 con 0x7f56040c9ad0 2023-10-07T23:21:54.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560a3e9700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 0x7f56040b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:54.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f5609be8700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f560000ec80 tx=0x7f56000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000c9d0 con 0x7f56040c9ad0 2023-10-07T23:21:54.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560000cb30 con 0x7f56040c9ad0 2023-10-07T23:21:54.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600013470 con 0x7f56040c9ad0 2023-10-07T23:21:54.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56040b0490 con 0x7f56040c9ad0 2023-10-07T23:21:54.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.006+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56041546a0 con 0x7f56040c9ad0 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56040711c0 con 0x7f56040c9ad0 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5600013820 con 0x7f56040c9ad0 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f55f5ffb700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 0x7f55e8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f560007cab0 con 0x7f56040c9ad0 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f56093e7700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 0x7f55e8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.010+0000 7f56093e7700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 0x7f55e8062a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5604064b60 tx=0x7f55f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.018+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f560004d0a0 con 0x7f56040c9ad0 2023-10-07T23:21:54.186 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.182+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5604063030 con 0x7f56040c9ad0 2023-10-07T23:21:54.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f55f5ffb700 1 -- 172.21.15.57:0/1554119347 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f56000506c0 con 0x7f56040c9ad0 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 msgr2=0x7f55e8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 0x7f55e8062a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5604064b60 tx=0x7f55f8009040 comp rx=0 tx=0).stop 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 msgr2=0x7f5604154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f560000ec80 tx=0x7f56000091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 shutdown_connections 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f560415d080 0x7f56040b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f55e8060580 0x7f55e8062a30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5604158ce0 0x7f56040b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 --2- 172.21.15.57:0/1554119347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f56040c9ad0 0x7f5604154160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.190+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 >> 172.21.15.57:0/1554119347 conn(0x7f5604067b40 msgr2=0x7f5604155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:54.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.194+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 shutdown_connections 2023-10-07T23:21:54.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.194+0000 7f560abea700 1 -- 172.21.15.57:0/1554119347 wait complete. 2023-10-07T23:21:54.212 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T23:21:54.212 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:21:54.232 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:21:54.254 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T23:21:54.255 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-07T23:21:54.255 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:21:54.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T23:21:54.283 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T23:21:54.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:21:54.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:54.283 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T23:21:54.338 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-10-07T23:21:54.338 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:21:54.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T23:21:54.361 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T23:21:54.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:21:54.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:54.361 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T23:21:54.420 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-07T23:21:54.420 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:21:54.420 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-07T23:21:54.437 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T23:21:54.438 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:21:54.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:54.438 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T23:21:54.488 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-10-07T23:21:54.488 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:21:54.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-07T23:21:54.504 INFO:teuthology.orchestra.run:waiting for 900 2023-10-07T23:21:54.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:21:54.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:21:54.505 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T23:21:54.554 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-07T23:21:54.555 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:54.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 Processor -- start 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 -- start start 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 0x7f5f8c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5f8c0b04c0 0x7f5f8c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cb9f0 con 0x7f5f8c158ef0 2023-10-07T23:21:54.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c06ace0 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f91037700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cb430 con 0x7f5f8c1592c0 2023-10-07T23:21:54.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8ad9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 0x7f5f8c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8ad9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 0x7f5f8c0aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50808/0 (socket says 172.21.15.57:50808) 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8ad9d700 1 -- 172.21.15.57:0/1860412761 learned_addr learned my addr 172.21.15.57:0/1860412761 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/1860412761 >> v2:172.21.15.154:3300/0 conn(0x7f5f8c0b04c0 0x7f5f8c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8b59e700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 -- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 msgr2=0x7f5f8c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 -- 172.21.15.57:0/1860412761 >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 msgr2=0x7f5f8c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/1860412761 >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 0x7f5f8c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 -- 172.21.15.57:0/1860412761 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0cb710 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c06c800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c068fd0 tx=0x7f5f7c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bef7b897f851f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f89d9b700 1 -- 172.21.15.57:0/1860412761 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00e070 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f89d9b700 1 -- 172.21.15.57:0/1860412761 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c00bab0 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.666+0000 7f5f89d9b700 1 -- 172.21.15.57:0/1860412761 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c00be50 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c0b04c0 msgr2=0x7f5f8c06c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c06c800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c068fd0 tx=0x7f5f7c009780 comp rx=0 tx=0).stop 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- 172.21.15.57:0/1860412761 shutdown_connections 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- 172.21.15.57:0/1860412761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c06c800 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- 172.21.15.57:0/1860412761 >> v2:172.21.15.57:3300/0 conn(0x7f5f8c158ef0 0x7f5f8c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- 172.21.15.57:0/1860412761 >> 172.21.15.57:0/1860412761 conn(0x7f5f8c0684c0 msgr2=0x7f5f8c0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- 172.21.15.57:0/1860412761 shutdown_connections 2023-10-07T23:21:54.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- 172.21.15.57:0/1860412761 wait complete. 2023-10-07T23:21:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 Processor -- start 2023-10-07T23:21:54.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- start start 2023-10-07T23:21:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c158ef0 0x7f5f8c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0b5c80 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0b5470 con 0x7f5f8c158ef0 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.670+0000 7f5f91037700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cef40 con 0x7f5f8c1592c0 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50822/0 (socket says 172.21.15.57:50822) 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 -- 172.21.15.57:0/3010598337 learned_addr learned my addr 172.21.15.57:0/3010598337 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c158ef0 0x7f5f8c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8b59e700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 -- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 msgr2=0x7f5f8c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 -- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c158ef0 msgr2=0x7f5f8c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c158ef0 0x7f5f8c0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 -- 172.21.15.57:0/3010598337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c009a20 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f8ad9d700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e980 tx=0x7f5f78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78013680 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780137e0 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c0c3ab0 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78013b60 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c0b59a0 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.674+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c0c3910 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.678+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5f7800c400 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.678+0000 7f5f82ffd700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 0x7f5f74062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:54.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.678+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f7807c870 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.678+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 0x7f5f74062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:54.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.678+0000 7f5f8a59c700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 0x7f5f74062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009fd0 tx=0x7f5f7c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:54.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.686+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5f8c0c3910 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.886+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f8c0c3910 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.886+0000 7f5f82ffd700 1 -- 172.21.15.57:0/3010598337 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f5f8c0c3910 con 0x7f5f8c0b04c0 2023-10-07T23:21:54.892 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:54.892 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:54.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 msgr2=0x7f5f74062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 0x7f5f74062a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009fd0 tx=0x7f5f7c019000 comp rx=0 tx=0).stop 2023-10-07T23:21:54.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 msgr2=0x7f5f8c1505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800e980 tx=0x7f5f78009250 comp rx=0 tx=0).stop 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 shutdown_connections 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5f8c1592c0 0x7f5f8c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5f74060580 0x7f5f74062a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f8c158ef0 0x7f5f8c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 --2- 172.21.15.57:0/3010598337 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f8c0b04c0 0x7f5f8c1505f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 >> 172.21.15.57:0/3010598337 conn(0x7f5f8c0684c0 msgr2=0x7f5f8c1567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 shutdown_connections 2023-10-07T23:21:54.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:54.890+0000 7f5f91037700 1 -- 172.21.15.57:0/3010598337 wait complete. 2023-10-07T23:21:54.896 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:54.906 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:55.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 Processor -- start 2023-10-07T23:21:55.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 -- start start 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f05ac151090 0x7f05ac15a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 0x7f05ac0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac06ae80 con 0x7f05ac0c8750 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0c8bd0 con 0x7f05ac151090 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05ab59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac15b050 con 0x7f05ac157d90 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05aa59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f05ac151090 0x7f05ac15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05aa59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f05ac151090 0x7f05ac15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50832/0 (socket says 172.21.15.57:50832) 2023-10-07T23:21:55.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.006+0000 7f05aa59c700 1 -- 172.21.15.57:0/1208423349 learned_addr learned my addr 172.21.15.57:0/1208423349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05a9d9b700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 0x7f05ac0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05aa59c700 1 -- 172.21.15.57:0/1208423349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 msgr2=0x7f05ac0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05aa59c700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 0x7f05ac0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05aa59c700 1 -- 172.21.15.57:0/1208423349 >> v1:172.21.15.57:6789/0 conn(0x7f05ac0c8750 legacy=0x7f05ac06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05aa59c700 1 -- 172.21.15.57:0/1208423349 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac15b330 con 0x7f05ac151090 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05a9d9b700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 0x7f05ac0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:21:55.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05aa59c700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac15a750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009f20 tx=0x7f05a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6efa4f662b35020 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05a959a700 1 -- 172.21.15.57:0/1208423349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400e070 con 0x7f05ac151090 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05a959a700 1 -- 172.21.15.57:0/1208423349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400ba80 con 0x7f05ac151090 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05a959a700 1 -- 172.21.15.57:0/1208423349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40092e0 con 0x7f05ac151090 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 -- 172.21.15.57:0/1208423349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 msgr2=0x7f05ac15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac15a750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009f20 tx=0x7f05a4009840 comp rx=0 tx=0).stop 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 -- 172.21.15.57:0/1208423349 shutdown_connections 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac157d90 0x7f05ac0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 --2- 172.21.15.57:0/1208423349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac15a750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 -- 172.21.15.57:0/1208423349 >> 172.21.15.57:0/1208423349 conn(0x7f05ac0670e0 msgr2=0x7f05ac0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 -- 172.21.15.57:0/1208423349 shutdown_connections 2023-10-07T23:21:55.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 -- 172.21.15.57:0/1208423349 wait complete. 2023-10-07T23:21:55.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.010+0000 7f05ab59e700 1 Processor -- start 2023-10-07T23:21:55.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- start start 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac0c8750 0x7f05ac143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ac157d90 0x7f05ac0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0c3be0 con 0x7f05ac151090 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0c3750 con 0x7f05ac157d90 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac143950 con 0x7f05ac0c8750 2023-10-07T23:21:55.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05aa59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac0c8750 0x7f05ac143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50836/0 (socket says 172.21.15.57:50836) 2023-10-07T23:21:55.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 -- 172.21.15.57:0/3034593827 learned_addr learned my addr 172.21.15.57:0/3034593827 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:55.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05aad9d700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ac157d90 0x7f05ac0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 -- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac0c8750 msgr2=0x7f05ac143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac0c8750 0x7f05ac143410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 -- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ac157d90 msgr2=0x7f05ac0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ac157d90 0x7f05ac0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 -- 172.21.15.57:0/3034593827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009ae0 con 0x7f05ac151090 2023-10-07T23:21:55.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a9d9b700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f059400c850 tx=0x7f0594008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011600 con 0x7f05ac151090 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0c2b40 con 0x7f05ac151090 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594011760 con 0x7f05ac151090 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011b00 con 0x7f05ac151090 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.014+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0c3100 con 0x7f05ac151090 2023-10-07T23:21:55.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0c38c0 con 0x7f05ac151090 2023-10-07T23:21:55.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0594018350 con 0x7f05ac151090 2023-10-07T23:21:55.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05a27fc700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f059407c730 con 0x7f05ac151090 2023-10-07T23:21:55.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05aa59c700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.018+0000 7f05aa59c700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009880 tx=0x7f05a4018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.026+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0594050710 con 0x7f05ac151090 2023-10-07T23:21:55.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.182+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05ac0711c0 con 0x7f05ac151090 2023-10-07T23:21:55.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05a27fc700 1 -- 172.21.15.57:0/3034593827 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f05940502a0 con 0x7f05ac151090 2023-10-07T23:21:55.190 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:55.190 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 msgr2=0x7f058c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009880 tx=0x7f05a4018000 comp rx=0 tx=0).stop 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 msgr2=0x7f05ac0b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f059400c850 tx=0x7f0594008040 comp rx=0 tx=0).stop 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 shutdown_connections 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ac157d90 0x7f05ac0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ac151090 0x7f05ac0b0c40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 --2- 172.21.15.57:0/3034593827 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05ac0c8750 0x7f05ac143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 >> 172.21.15.57:0/3034593827 conn(0x7f05ac0670e0 msgr2=0x7f05ac0ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 shutdown_connections 2023-10-07T23:21:55.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.186+0000 7f05ab59e700 1 -- 172.21.15.57:0/3034593827 wait complete. 2023-10-07T23:21:55.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:21:22.957393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:21:24.292476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:21:55.204 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:21:55.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.314+0000 7fee1ffff700 1 Processor -- start 2023-10-07T23:21:55.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.314+0000 7fee1ffff700 1 -- start start 2023-10-07T23:21:55.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fee200abc40 0x7fee200cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fee200ccf50 0x7fee200697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1ffff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee20069350 con 0x7fee20069d20 2023-10-07T23:21:55.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1ffff700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200cd9b0 con 0x7fee200abc40 2023-10-07T23:21:55.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1ffff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200cd3f0 con 0x7fee200ccf50 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fee200abc40 0x7fee200cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1f7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fee20069d20 0x7fee2006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48308/0 (socket says 172.21.15.57:48308) 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1f7fe700 1 -- 172.21.15.57:0/3382874708 learned_addr learned my addr 172.21.15.57:0/3382874708 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee17fff700 1 --2- 172.21.15.57:0/3382874708 >> v2:172.21.15.154:3300/0 conn(0x7fee200ccf50 0x7fee200697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1e7fc700 1 -- 172.21.15.57:0/3382874708 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334923246 0 0) 0x7fee20069350 con 0x7fee20069d20 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 -- 172.21.15.57:0/3382874708 >> v2:172.21.15.154:3300/0 conn(0x7fee200ccf50 msgr2=0x7fee200697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 --2- 172.21.15.57:0/3382874708 >> v2:172.21.15.154:3300/0 conn(0x7fee200ccf50 0x7fee200697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 -- 172.21.15.57:0/3382874708 >> v1:172.21.15.57:6789/0 conn(0x7fee20069d20 legacy=0x7fee2006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 -- 172.21.15.57:0/3382874708 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200cd6d0 con 0x7fee200abc40 2023-10-07T23:21:55.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1effd700 1 --2- 172.21.15.57:0/3382874708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee200cca10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee2015a730 tx=0x7fee100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0282ba8f9f6dbbc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1e7fc700 1 -- 172.21.15.57:0/3382874708 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10011630 con 0x7fee200abc40 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1e7fc700 1 -- 172.21.15.57:0/3382874708 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10011790 con 0x7fee200abc40 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.318+0000 7fee1e7fc700 1 -- 172.21.15.57:0/3382874708 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10011b30 con 0x7fee200abc40 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- 172.21.15.57:0/3382874708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 msgr2=0x7fee200cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 --2- 172.21.15.57:0/3382874708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee200cca10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee2015a730 tx=0x7fee100091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- 172.21.15.57:0/3382874708 shutdown_connections 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 --2- 172.21.15.57:0/3382874708 >> v2:172.21.15.154:3300/0 conn(0x7fee200ccf50 0x7fee200697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 --2- 172.21.15.57:0/3382874708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee200cca10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- 172.21.15.57:0/3382874708 >> 172.21.15.57:0/3382874708 conn(0x7fee2006b5d0 msgr2=0x7fee2006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- 172.21.15.57:0/3382874708 shutdown_connections 2023-10-07T23:21:55.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- 172.21.15.57:0/3382874708 wait complete. 2023-10-07T23:21:55.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 Processor -- start 2023-10-07T23:21:55.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.322+0000 7fee1ffff700 1 -- start start 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee20069d20 0x7fee200ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee200ccf50 0x7fee200c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200ad880 con 0x7fee200abc40 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20157bb0 con 0x7fee200ccf50 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200acc70 con 0x7fee20069d20 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50864/0 (socket says 172.21.15.57:50864) 2023-10-07T23:21:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 -- 172.21.15.57:0/274267433 learned_addr learned my addr 172.21.15.57:0/274267433 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1effd700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee20069d20 0x7fee200ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1f7fe700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee200ccf50 0x7fee200c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 -- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee20069d20 msgr2=0x7fee200ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee20069d20 0x7fee200ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 -- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee200ccf50 msgr2=0x7fee200c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee200ccf50 0x7fee200c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 -- 172.21.15.57:0/274267433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee10009420 con 0x7fee200abc40 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee17fff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee0800e930 tx=0x7fee080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08013680 con 0x7fee200abc40 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee080137e0 con 0x7fee200abc40 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08013b80 con 0x7fee200abc40 2023-10-07T23:21:55.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee200acf50 con 0x7fee200abc40 2023-10-07T23:21:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.326+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee200ad510 con 0x7fee200abc40 2023-10-07T23:21:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee200ada30 con 0x7fee200abc40 2023-10-07T23:21:55.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fee0800c3e0 con 0x7fee200abc40 2023-10-07T23:21:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee16ffd700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 0x7fee00062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee0807c860 con 0x7fee200abc40 2023-10-07T23:21:55.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee1effd700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 0x7fee00062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.330+0000 7fee1effd700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 0x7fee00062a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee100093f0 tx=0x7fee10017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.338+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fee200ada30 con 0x7fee200abc40 2023-10-07T23:21:55.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.498+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee200711c0 con 0x7fee200abc40 2023-10-07T23:21:55.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.498+0000 7fee16ffd700 1 -- 172.21.15.57:0/274267433 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee0804cdb0 con 0x7fee200abc40 2023-10-07T23:21:55.504 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:21:55.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 msgr2=0x7fee00062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 0x7fee00062a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fee100093f0 tx=0x7fee10017000 comp rx=0 tx=0).stop 2023-10-07T23:21:55.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 msgr2=0x7fee2014a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fee0800e930 tx=0x7fee080091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 shutdown_connections 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fee00060580 0x7fee00062a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee200ccf50 0x7fee200c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee200abc40 0x7fee2014a8e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 --2- 172.21.15.57:0/274267433 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fee20069d20 0x7fee200ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 >> 172.21.15.57:0/274267433 conn(0x7fee2006b5d0 msgr2=0x7fee2014b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 shutdown_connections 2023-10-07T23:21:55.507 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.502+0000 7fee1ffff700 1 -- 172.21.15.57:0/274267433 wait complete. 2023-10-07T23:21:55.524 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:21:55.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 Processor -- start 2023-10-07T23:21:55.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 -- start start 2023-10-07T23:21:55.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8fc406cf10 0x7f8fc40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc406a900 con 0x7f8fc4159030 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc406a600 con 0x7f8fc40ca600 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fcc5d9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40b0450 con 0x7f8fc406cf10 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9b74700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8fc406cf10 0x7f8fc40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fca375700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8fc4159030 0x7f8fc406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48316/0 (socket says 172.21.15.57:48316) 2023-10-07T23:21:55.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fca375700 1 -- 172.21.15.57:0/1383249595 learned_addr learned my addr 172.21.15.57:0/1383249595 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9373700 1 -- 172.21.15.57:0/1383249595 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015984753 0 0) 0x7f8fc406a900 con 0x7f8fc4159030 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9373700 1 -- 172.21.15.57:0/1383249595 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb0003570 con 0x7f8fc4159030 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9b74700 1 -- 172.21.15.57:0/1383249595 >> v1:172.21.15.154:6789/0 conn(0x7f8fc40ca600 legacy=0x7f8fc40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9b74700 1 -- 172.21.15.57:0/1383249595 >> v1:172.21.15.57:6789/0 conn(0x7f8fc4159030 legacy=0x7f8fc406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.634+0000 7f8fc9b74700 1 -- 172.21.15.57:0/1383249595 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40cb350 con 0x7f8fc406cf10 2023-10-07T23:21:55.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fc9b74700 1 --2- 172.21.15.57:0/1383249595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc406cf10 0x7f8fc40aff10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098e0 tx=0x7f8fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0ff6a0fbd26016 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fc9373700 1 -- 172.21.15.57:0/1383249595 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400b640 con 0x7f8fc406cf10 2023-10-07T23:21:55.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fc9373700 1 -- 172.21.15.57:0/1383249595 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400b7a0 con 0x7f8fc406cf10 2023-10-07T23:21:55.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fc9373700 1 -- 172.21.15.57:0/1383249595 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400bb20 con 0x7f8fc406cf10 2023-10-07T23:21:55.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/1383249595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc406cf10 msgr2=0x7f8fc40aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/1383249595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc406cf10 0x7f8fc40aff10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098e0 tx=0x7f8fb40091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/1383249595 shutdown_connections 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/1383249595 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc406cf10 0x7f8fc40aff10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/1383249595 >> 172.21.15.57:0/1383249595 conn(0x7f8fc4068650 msgr2=0x7f8fc4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/1383249595 shutdown_connections 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/1383249595 wait complete. 2023-10-07T23:21:55.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 Processor -- start 2023-10-07T23:21:55.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.638+0000 7f8fcc5d9700 1 -- start start 2023-10-07T23:21:55.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fc406cf10 0x7f8fc40c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc40ca600 0x7f8fc41503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c9ae0 con 0x7f8fc4159030 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c8b80 con 0x7f8fc40ca600 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b3270 con 0x7f8fc406cf10 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fc9b74700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc40ca600 0x7f8fc41503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50878/0 (socket says 172.21.15.57:50878) 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 -- 172.21.15.57:0/2851106051 learned_addr learned my addr 172.21.15.57:0/2851106051 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:55.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fca375700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fc406cf10 0x7f8fc40c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 -- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fc406cf10 msgr2=0x7f8fc40c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fc406cf10 0x7f8fc40c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 -- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc40ca600 msgr2=0x7f8fc41503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc40ca600 0x7f8fc41503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 -- 172.21.15.57:0/2851106051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4009420 con 0x7f8fc4159030 2023-10-07T23:21:55.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcab76700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c560 tx=0x7f8fbc007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011590 con 0x7f8fc4159030 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0116f0 con 0x7f8fc4159030 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40b3550 con 0x7f8fc4159030 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011ab0 con 0x7f8fc4159030 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.642+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40b3b10 con 0x7f8fc4159030 2023-10-07T23:21:55.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40b3d20 con 0x7f8fc4159030 2023-10-07T23:21:55.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fbc01b3b0 con 0x7f8fc4159030 2023-10-07T23:21:55.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fba7fc700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 0x7f8fa8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fbc07bc10 con 0x7f8fc4159030 2023-10-07T23:21:55.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fca375700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 0x7f8fa8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.646+0000 7f8fca375700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 0x7f8fa8062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005c50 tx=0x7f8fc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.654+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fbc04c160 con 0x7f8fc4159030 2023-10-07T23:21:55.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.814+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fc40647b0 con 0x7f8fc4159030 2023-10-07T23:21:55.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.814+0000 7f8fba7fc700 1 -- 172.21.15.57:0/2851106051 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fbc04f780 con 0x7f8fc4159030 2023-10-07T23:21:55.819 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:21:55.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 msgr2=0x7f8fa8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 0x7f8fa8062a00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005c50 tx=0x7f8fc0009040 comp rx=0 tx=0).stop 2023-10-07T23:21:55.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 msgr2=0x7f8fc4150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c560 tx=0x7f8fbc007ef0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 shutdown_connections 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fc4159030 0x7f8fc4150920 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa8060550 0x7f8fa8062a00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fc40ca600 0x7f8fc41503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 --2- 172.21.15.57:0/2851106051 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fc406cf10 0x7f8fc40c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 >> 172.21.15.57:0/2851106051 conn(0x7f8fc4068650 msgr2=0x7f8fc4152600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 shutdown_connections 2023-10-07T23:21:55.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.818+0000 7f8fcc5d9700 1 -- 172.21.15.57:0/2851106051 wait complete. 2023-10-07T23:21:55.834 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:21:55.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 Processor -- start 2023-10-07T23:21:55.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 -- start start 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f362815d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f362806a6f0 0x7f362806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36280ca530 con 0x7f3628158c40 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362806ae60 con 0x7f362806a6f0 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362effa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362806b140 con 0x7f3628159010 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362dff8700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3628158c40 0x7f36280afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48332/0 (socket says 172.21.15.57:48332) 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362dff8700 1 -- 172.21.15.57:0/3244391853 learned_addr learned my addr 172.21.15.57:0/3244391853 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362d7f7700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f362815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.942+0000 7f362e7f9700 1 --2- 172.21.15.57:0/3244391853 >> v2:172.21.15.154:3300/0 conn(0x7f362806a6f0 0x7f362806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362cff6700 1 -- 172.21.15.57:0/3244391853 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044509507 0 0) 0x7f36280ca530 con 0x7f3628158c40 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362cff6700 1 -- 172.21.15.57:0/3244391853 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3610003570 con 0x7f3628158c40 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362e7f9700 1 -- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 msgr2=0x7f362815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362e7f9700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f362815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362e7f9700 1 -- 172.21.15.57:0/3244391853 >> v1:172.21.15.57:6789/0 conn(0x7f3628158c40 legacy=0x7f36280afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362e7f9700 1 -- 172.21.15.57:0/3244391853 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362806b420 con 0x7f362806a6f0 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362d7f7700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f362815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362e7f9700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362806c800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad82d50c9ad9ba48 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362cff6700 1 -- 172.21.15.57:0/3244391853 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618011630 con 0x7f362806a6f0 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362cff6700 1 -- 172.21.15.57:0/3244391853 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618011790 con 0x7f362806a6f0 2023-10-07T23:21:55.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362cff6700 1 -- 172.21.15.57:0/3244391853 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618011b10 con 0x7f362806a6f0 2023-10-07T23:21:55.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 -- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 msgr2=0x7f362806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362806c800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f36180098e0 tx=0x7f36180091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 -- 172.21.15.57:0/3244391853 shutdown_connections 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362806c800 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 --2- 172.21.15.57:0/3244391853 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f362815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 -- 172.21.15.57:0/3244391853 >> 172.21.15.57:0/3244391853 conn(0x7f3628062ba0 msgr2=0x7f3628062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 -- 172.21.15.57:0/3244391853 shutdown_connections 2023-10-07T23:21:55.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.946+0000 7f362effa700 1 -- 172.21.15.57:0/3244391853 wait complete. 2023-10-07T23:21:55.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 Processor -- start 2023-10-07T23:21:55.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 -- start start 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f3628159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3628153d50 con 0x7f3628158c40 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280ccfb0 con 0x7f362806a6f0 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362effa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3628153430 con 0x7f3628159010 2023-10-07T23:21:55.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362d7f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362d7f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50880/0 (socket says 172.21.15.57:50880) 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362d7f7700 1 -- 172.21.15.57:0/742475322 learned_addr learned my addr 172.21.15.57:0/742475322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362dff8700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.950+0000 7f362e7f9700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f3628159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 -- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 msgr2=0x7f3628159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f3628159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 -- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 msgr2=0x7f362815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 -- 172.21.15.57:0/742475322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c009a20 con 0x7f3628158c40 2023-10-07T23:21:55.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362e7f9700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f3628159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362d7f7700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c170 tx=0x7f361c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00cd60 con 0x7f3628158c40 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00f040 con 0x7f3628158c40 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618009420 con 0x7f3628158c40 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c014690 con 0x7f3628158c40 2023-10-07T23:21:55.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3628153700 con 0x7f3628158c40 2023-10-07T23:21:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36280711c0 con 0x7f3628158c40 2023-10-07T23:21:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f361c014a00 con 0x7f3628158c40 2023-10-07T23:21:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 0x7f3614062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.954+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f361c07d640 con 0x7f3628158c40 2023-10-07T23:21:55.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.958+0000 7f362dff8700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 0x7f3614062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:55.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.958+0000 7f362dff8700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 0x7f3614062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3620007fd0 tx=0x7f3620009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:55.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:55.962+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f361c04dc30 con 0x7f3628158c40 2023-10-07T23:21:56.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36281560e0 con 0x7f3628158c40 2023-10-07T23:21:56.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f3625ffb700 1 -- 172.21.15.57:0/742475322 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f361c014cb0 con 0x7f3628158c40 2023-10-07T23:21:56.127 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 msgr2=0x7f3614062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 0x7f3614062a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3620007fd0 tx=0x7f3620009040 comp rx=0 tx=0).stop 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 msgr2=0x7f3628159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c170 tx=0x7f361c00b040 comp rx=0 tx=0).stop 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 shutdown_connections 2023-10-07T23:21:56.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3628159010 0x7f3628159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3628158c40 0x7f3628159720 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3614060550 0x7f3614062a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 --2- 172.21.15.57:0/742475322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f362806a6f0 0x7f362815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 >> 172.21.15.57:0/742475322 conn(0x7f3628062ba0 msgr2=0x7f36280b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 shutdown_connections 2023-10-07T23:21:56.130 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.122+0000 7f362effa700 1 -- 172.21.15.57:0/742475322 wait complete. 2023-10-07T23:21:56.142 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:56.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 Processor -- start 2023-10-07T23:21:56.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- start start 2023-10-07T23:21:56.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffa3806bc90 0x7ffa380c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38159130 0x7ffa380c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa380c83b0 con 0x7ffa38158d60 2023-10-07T23:21:56.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa380c8690 con 0x7ffa3806bc90 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380c8970 con 0x7ffa38159130 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffa3806bc90 0x7ffa380c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ffa3806bc90 0x7ffa380c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50890/0 (socket says 172.21.15.57:50890) 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 -- 172.21.15.57:0/3165660133 learned_addr learned my addr 172.21.15.57:0/3165660133 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa37fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ffa38158d60 0x7ffa3806b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48342/0 (socket says 172.21.15.57:48342) 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3cf23700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38159130 0x7ffa380c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 -- 172.21.15.57:0/3165660133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38159130 msgr2=0x7ffa380c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38159130 0x7ffa380c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 -- 172.21.15.57:0/3165660133 >> v1:172.21.15.57:6789/0 conn(0x7ffa38158d60 legacy=0x7ffa3806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 -- 172.21.15.57:0/3165660133 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380c8c50 con 0x7ffa3806bc90 2023-10-07T23:21:56.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa377fe700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa380c7580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0099b0 tx=0x7ffa2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ada05d154024d71 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa36ffd700 1 -- 172.21.15.57:0/3165660133 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00b860 con 0x7ffa3806bc90 2023-10-07T23:21:56.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa36ffd700 1 -- 172.21.15.57:0/3165660133 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c00b9c0 con 0x7ffa3806bc90 2023-10-07T23:21:56.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa36ffd700 1 -- 172.21.15.57:0/3165660133 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00bd60 con 0x7ffa3806bc90 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- 172.21.15.57:0/3165660133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 msgr2=0x7ffa380c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa380c7580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0099b0 tx=0x7ffa2c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- 172.21.15.57:0/3165660133 shutdown_connections 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38159130 0x7ffa380c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 --2- 172.21.15.57:0/3165660133 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa380c7580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- 172.21.15.57:0/3165660133 >> 172.21.15.57:0/3165660133 conn(0x7ffa380670e0 msgr2=0x7ffa380674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- 172.21.15.57:0/3165660133 shutdown_connections 2023-10-07T23:21:56.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.250+0000 7ffa3d724700 1 -- 172.21.15.57:0/3165660133 wait complete. 2023-10-07T23:21:56.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 Processor -- start 2023-10-07T23:21:56.259 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- start start 2023-10-07T23:21:56.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38158d60 0x7ffa3814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa38159130 0x7ffa38155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3814c350 con 0x7ffa3806bc90 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380c9c90 con 0x7ffa38159130 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa380c9f70 con 0x7ffa38158d60 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50900/0 (socket says 172.21.15.57:50900) 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 -- 172.21.15.57:0/1049305119 learned_addr learned my addr 172.21.15.57:0/1049305119 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa377fe700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38158d60 0x7ffa3814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3cf23700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa38159130 0x7ffa38155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 -- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38158d60 msgr2=0x7ffa3814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38158d60 0x7ffa3814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 -- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa38159130 msgr2=0x7ffa38155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa38159130 0x7ffa38155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 -- 172.21.15.57:0/1049305119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0094f0 con 0x7ffa3806bc90 2023-10-07T23:21:56.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa37fff700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c7c0 tx=0x7ffa28007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28005a70 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa28005bd0 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa380ca250 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa280070d0 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.254+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa380ca810 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ffa280074c0 con 0x7ffa3806bc90 2023-10-07T23:21:56.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa380caa00 con 0x7ffa3806bc90 2023-10-07T23:21:56.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa1ffff700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 0x7ffa24062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa2807bb60 con 0x7ffa3806bc90 2023-10-07T23:21:56.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa377fe700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 0x7ffa24062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.258+0000 7ffa377fe700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 0x7ffa24062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0099b0 tx=0x7ffa2c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.266+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffa2804c150 con 0x7ffa3806bc90 2023-10-07T23:21:56.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.462+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa380711c0 con 0x7ffa3806bc90 2023-10-07T23:21:56.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa1ffff700 1 -- 172.21.15.57:0/1049305119 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7ffa2804f770 con 0x7ffa3806bc90 2023-10-07T23:21:56.470 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:56.470 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 msgr2=0x7ffa24062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 0x7ffa24062a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0099b0 tx=0x7ffa2c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 msgr2=0x7ffa3814be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c7c0 tx=0x7ffa28007f00 comp rx=0 tx=0).stop 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 shutdown_connections 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ffa24060550 0x7ffa24062a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffa38159130 0x7ffa38155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ffa38158d60 0x7ffa3814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 --2- 172.21.15.57:0/1049305119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3806bc90 0x7ffa3814be10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 >> 172.21.15.57:0/1049305119 conn(0x7ffa380670e0 msgr2=0x7ffa3814fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 shutdown_connections 2023-10-07T23:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.466+0000 7ffa3d724700 1 -- 172.21.15.57:0/1049305119 wait complete. 2023-10-07T23:21:56.474 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:56.485 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:56.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.586+0000 7f5d227da700 1 Processor -- start 2023-10-07T23:21:56.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.586+0000 7f5d227da700 1 -- start start 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d1c15ced0 0x7f5d1c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c06bbf0 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1c06b020 con 0x7f5d1c161a70 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c9010 con 0x7f5d1c15ced0 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d21fd9700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5d1c15d2a0 0x7f5d1c0c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60912/0 (socket says 172.21.15.57:60912) 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d21fd9700 1 -- 172.21.15.57:0/2224248281 learned_addr learned my addr 172.21.15.57:0/2224248281 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:56.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d217d8700 1 --2- 172.21.15.57:0/2224248281 >> v2:172.21.15.57:3300/0 conn(0x7f5d1c15ced0 0x7f5d1c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067223164 0 0) 0x7f5d1c06bbf0 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d08003570 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436065047 0 0) 0x7f5d08003570 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 >> v2:172.21.15.57:3300/0 conn(0x7f5d1c15ced0 msgr2=0x7f5d1c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 --2- 172.21.15.57:0/2224248281 >> v2:172.21.15.57:3300/0 conn(0x7f5d1c15ced0 0x7f5d1c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 >> v1:172.21.15.154:6789/0 conn(0x7f5d1c161a70 legacy=0x7f5d1c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c06b2c0 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f5d140017f0 con 0x7f5d1c15d2a0 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 >> v1:172.21.15.57:6789/0 conn(0x7f5d1c15d2a0 legacy=0x7f5d1c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08003a60 0x7f5d08003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 0x7f5d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d080051c0 0x7f5d0800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0c9010 con 0x7f5d08004810 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800c050 con 0x7f5d080051c0 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800c210 con 0x7f5d08003a60 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 0x7f5d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d217d8700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08003a60 0x7f5d08003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d21fd9700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d080051c0 0x7f5d0800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 -- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08003a60 msgr2=0x7f5d08003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08003a60 0x7f5d08003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 -- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d080051c0 msgr2=0x7f5d0800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d080051c0 0x7f5d0800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 -- 172.21.15.57:0/2224248281 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08003570 con 0x7f5d08004810 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d21fd9700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d080051c0 0x7f5d0800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d21fd9700 1 -- 172.21.15.57:0/2224248281 reap_dead start 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 0x7f5d08004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d100019a0 tx=0x7f5d10007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15f06ede6068ded server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000c5d0 con 0x7f5d08004810 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1000c730 con 0x7f5d08004810 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d0ffff700 1 -- 172.21.15.57:0/2224248281 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000cad0 con 0x7f5d08004810 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 msgr2=0x7f5d08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 0x7f5d08004c80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d100019a0 tx=0x7f5d10007d10 comp rx=0 tx=0).stop 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- 172.21.15.57:0/2224248281 shutdown_connections 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 --2- 172.21.15.57:0/2224248281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d08004810 0x7f5d08004c80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- 172.21.15.57:0/2224248281 >> 172.21.15.57:0/2224248281 conn(0x7f5d1c0670e0 msgr2=0x7f5d1c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- 172.21.15.57:0/2224248281 shutdown_connections 2023-10-07T23:21:56.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.590+0000 7f5d227da700 1 -- 172.21.15.57:0/2224248281 wait complete. 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 Processor -- start 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- start start 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08004810 0x7f5d1c142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c0b06d0 0x7f5d1c0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c143040 con 0x7f5d1c0afa80 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c0b0430 con 0x7f5d1c0b06d0 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1c15f1f0 con 0x7f5d08004810 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33428/0 (socket says 172.21.15.57:33428) 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 -- 172.21.15.57:0/3115355787 learned_addr learned my addr 172.21.15.57:0/3115355787 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d217d8700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08004810 0x7f5d1c142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d21fd9700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c0b06d0 0x7f5d1c0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 -- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08004810 msgr2=0x7f5d1c142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08004810 0x7f5d1c142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 -- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c0b06d0 msgr2=0x7f5d1c0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c0b06d0 0x7f5d1c0b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 -- 172.21.15.57:0/3115355787 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c15e180 con 0x7f5d1c0afa80 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d20fd7700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d10012ef0 tx=0x7f5d10015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000c5d0 con 0x7f5d1c0afa80 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1000ce40 con 0x7f5d1c0afa80 2023-10-07T23:21:56.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1001e400 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c15e430 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.594+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c15e9f0 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1c063750 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5d10019030 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d0d7fa700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 0x7f5d00062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d1007dd60 con 0x7f5d1c0afa80 2023-10-07T23:21:56.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d217d8700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 0x7f5d00062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.598+0000 7f5d217d8700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 0x7f5d00062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009ee0 tx=0x7f5d18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.610 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.606+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d1c063750 con 0x7f5d1c0afa80 2023-10-07T23:21:56.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.766+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d1c063750 con 0x7f5d1c0afa80 2023-10-07T23:21:56.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d0d7fa700 1 -- 172.21.15.57:0/3115355787 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f5d1c063750 con 0x7f5d1c0afa80 2023-10-07T23:21:56.774 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:56.774 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 msgr2=0x7f5d00062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 0x7f5d00062a00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009ee0 tx=0x7f5d18008040 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 msgr2=0x7f5d1c0afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d10012ef0 tx=0x7f5d10015000 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 shutdown_connections 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d00060550 0x7f5d00062a00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d1c0b06d0 0x7f5d1c0b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d1c0afa80 0x7f5d1c0afef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 --2- 172.21.15.57:0/3115355787 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d08004810 0x7f5d1c142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 >> 172.21.15.57:0/3115355787 conn(0x7f5d1c0670e0 msgr2=0x7f5d1c159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 shutdown_connections 2023-10-07T23:21:56.777 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.770+0000 7f5d227da700 1 -- 172.21.15.57:0/3115355787 wait complete. 2023-10-07T23:21:56.788 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T23:21:56.789 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:56.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 Processor -- start 2023-10-07T23:21:56.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 -- start start 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f47900dfde0 0x7f47900e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 0x7f479004fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4790037ae0 0x7f47900e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47900503d0 con 0x7f47900dfde0 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47900506b0 con 0x7f4790037ae0 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f479ca72700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4790050990 con 0x7f4790051470 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f47900dfde0 0x7f47900e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f47900dfde0 0x7f47900e01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33436/0 (socket says 172.21.15.57:33436) 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 -- 172.21.15.57:0/752072994 learned_addr learned my addr 172.21.15.57:0/752072994 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f4796ffd700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 0x7f479004fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f4797fff700 1 --2- 172.21.15.57:0/752072994 >> v2:172.21.15.154:3300/0 conn(0x7f4790037ae0 0x7f47900e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 -- 172.21.15.57:0/752072994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 msgr2=0x7f479004fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 0x7f479004fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 -- 172.21.15.57:0/752072994 >> v2:172.21.15.154:3300/0 conn(0x7f4790037ae0 msgr2=0x7f47900e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 --2- 172.21.15.57:0/752072994 >> v2:172.21.15.154:3300/0 conn(0x7f4790037ae0 0x7f47900e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.890+0000 7f47977fe700 1 -- 172.21.15.57:0/752072994 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790050c70 con 0x7f47900dfde0 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f4796ffd700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 0x7f479004fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:56.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f47977fe700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47900dfde0 0x7f47900e01b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f479000e4e0 tx=0x7f478c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d6c4f60505fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f47967fc700 1 -- 172.21.15.57:0/752072994 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00b860 con 0x7f47900dfde0 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f47967fc700 1 -- 172.21.15.57:0/752072994 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c00b9c0 con 0x7f47900dfde0 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f47967fc700 1 -- 172.21.15.57:0/752072994 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00bd60 con 0x7f47900dfde0 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 -- 172.21.15.57:0/752072994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47900dfde0 msgr2=0x7f47900e01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47900dfde0 0x7f47900e01b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f479000e4e0 tx=0x7f478c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 -- 172.21.15.57:0/752072994 shutdown_connections 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 --2- 172.21.15.57:0/752072994 >> v2:172.21.15.154:3300/0 conn(0x7f4790037ae0 0x7f47900e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4790051470 0x7f479004fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 --2- 172.21.15.57:0/752072994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f47900dfde0 0x7f47900e01b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 -- 172.21.15.57:0/752072994 >> 172.21.15.57:0/752072994 conn(0x7f4790052100 msgr2=0x7f4790052500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 -- 172.21.15.57:0/752072994 shutdown_connections 2023-10-07T23:21:56.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.894+0000 7f479ca72700 1 -- 172.21.15.57:0/752072994 wait complete. 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 Processor -- start 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- start start 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4790051470 0x7f479004c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47900dfde0 0x7f47900cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479004d7a0 con 0x7f4790037ae0 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47900e0940 con 0x7f4790051470 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479004cc80 con 0x7f47900dfde0 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33448/0 (socket says 172.21.15.57:33448) 2023-10-07T23:21:56.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 -- 172.21.15.57:0/2983928690 learned_addr learned my addr 172.21.15.57:0/2983928690 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f4797fff700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47900dfde0 0x7f47900cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f4796ffd700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4790051470 0x7f479004c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 -- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47900dfde0 msgr2=0x7f47900cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47900dfde0 0x7f47900cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 -- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4790051470 msgr2=0x7f479004c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4790051470 0x7f479004c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 -- 172.21.15.57:0/2983928690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c009490 con 0x7f4790037ae0 2023-10-07T23:21:56.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47977fe700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f478c0090b0 tx=0x7f478c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c00b860 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c01f070 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c026480 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47900cb630 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.898+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479004d1e0 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4790005950 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f478c01a430 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f47837fe700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 0x7f477c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f478c021070 con 0x7f4790037ae0 2023-10-07T23:21:56.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f4796ffd700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 0x7f477c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:56.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.902+0000 7f4796ffd700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 0x7f477c062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f47900e4a10 tx=0x7f4784008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:56.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:56.910+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f478c04e9d0 con 0x7f4790037ae0 2023-10-07T23:21:57.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.110+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4790051960 con 0x7f4790037ae0 2023-10-07T23:21:57.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.110+0000 7f47837fe700 1 -- 172.21.15.57:0/2983928690 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f478c051ff0 con 0x7f4790037ae0 2023-10-07T23:21:57.115 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:57.115 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:57.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 msgr2=0x7f477c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 0x7f477c062a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f47900e4a10 tx=0x7f4784008040 comp rx=0 tx=0).stop 2023-10-07T23:21:57.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 msgr2=0x7f47900e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f478c0090b0 tx=0x7f478c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 shutdown_connections 2023-10-07T23:21:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f47900dfde0 0x7f47900cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f477c060550 0x7f477c062a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4790051470 0x7f479004c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 --2- 172.21.15.57:0/2983928690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4790037ae0 0x7f47900e44d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 >> 172.21.15.57:0/2983928690 conn(0x7f4790052100 msgr2=0x7f47900d92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 shutdown_connections 2023-10-07T23:21:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.114+0000 7f479ca72700 1 -- 172.21.15.57:0/2983928690 wait complete. 2023-10-07T23:21:57.121 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:57.135 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:21:57.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 Processor -- start 2023-10-07T23:21:57.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 -- start start 2023-10-07T23:21:57.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 0x7efc5c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c06acc0 con 0x7efc5c06cd40 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c06d1e0 con 0x7efc5c156440 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc62b5e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0c8b10 con 0x7efc5c0cadd0 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56694/0 (socket says 172.21.15.57:56694) 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 -- 172.21.15.57:0/3705548366 learned_addr learned my addr 172.21.15.57:0/3705548366 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc6135b700 1 --2- 172.21.15.57:0/3705548366 >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 0x7efc5c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 -- 172.21.15.57:0/3705548366 >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 msgr2=0x7efc5c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 --2- 172.21.15.57:0/3705548366 >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 0x7efc5c06c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 -- 172.21.15.57:0/3705548366 >> v1:172.21.15.154:6789/0 conn(0x7efc5c06cd40 legacy=0x7efc5c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 -- 172.21.15.57:0/3705548366 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0c80f0 con 0x7efc5c0cadd0 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc6135b700 1 --2- 172.21.15.57:0/3705548366 >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 0x7efc5c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.238+0000 7efc61b5c700 1 --2- 172.21.15.57:0/3705548366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0099b0 tx=0x7efc4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3968a3d22414d089 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc60b5a700 1 -- 172.21.15.57:0/3705548366 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c009ea0 con 0x7efc5c0cadd0 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc60b5a700 1 -- 172.21.15.57:0/3705548366 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00b860 con 0x7efc5c0cadd0 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc60b5a700 1 -- 172.21.15.57:0/3705548366 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00bc00 con 0x7efc5c0cadd0 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- 172.21.15.57:0/3705548366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 msgr2=0x7efc5c0c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- 172.21.15.57:0/3705548366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0099b0 tx=0x7efc4c009250 comp rx=0 tx=0).stop 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- 172.21.15.57:0/3705548366 shutdown_connections 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- 172.21.15.57:0/3705548366 >> v2:172.21.15.154:3300/0 conn(0x7efc5c156440 0x7efc5c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- 172.21.15.57:0/3705548366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c0cadd0 0x7efc5c0c77c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- 172.21.15.57:0/3705548366 >> 172.21.15.57:0/3705548366 conn(0x7efc5c0670e0 msgr2=0x7efc5c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- 172.21.15.57:0/3705548366 shutdown_connections 2023-10-07T23:21:57.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- 172.21.15.57:0/3705548366 wait complete. 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 Processor -- start 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- start start 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c0cadd0 0x7efc5c0af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c156440 0x7efc5c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c169330 con 0x7efc5c06cd40 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0afd30 con 0x7efc5c0cadd0 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc62b5e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0b0010 con 0x7efc5c156440 2023-10-07T23:21:57.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc61b5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc61b5c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33456/0 (socket says 172.21.15.57:33456) 2023-10-07T23:21:57.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc61b5c700 1 -- 172.21.15.57:0/2100856668 learned_addr learned my addr 172.21.15.57:0/2100856668 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc6235d700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c156440 0x7efc5c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc6135b700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c0cadd0 0x7efc5c0af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.242+0000 7efc61b5c700 1 -- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c156440 msgr2=0x7efc5c0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc61b5c700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c156440 0x7efc5c0af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc61b5c700 1 -- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c0cadd0 msgr2=0x7efc5c0af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc61b5c700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c0cadd0 0x7efc5c0af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc61b5c700 1 -- 172.21.15.57:0/2100856668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009a20 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc61b5c700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc5c14fdd0 tx=0x7efc4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c000d10 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00bbe0 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c0094f0 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c011670 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c0b05d0 con 0x7efc5c06cd40 2023-10-07T23:21:57.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c0711c0 con 0x7efc5c06cd40 2023-10-07T23:21:57.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.246+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efc4c011aa0 con 0x7efc5c06cd40 2023-10-07T23:21:57.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.250+0000 7efc59ffb700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 0x7efc48062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.250+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc4c07dff0 con 0x7efc5c06cd40 2023-10-07T23:21:57.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.250+0000 7efc6135b700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 0x7efc48062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.250+0000 7efc6135b700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 0x7efc48062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc50009fd0 tx=0x7efc5000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.254+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efc4c0adbe0 con 0x7efc5c06cd40 2023-10-07T23:21:57.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.450+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc5c0c59d0 con 0x7efc5c06cd40 2023-10-07T23:21:57.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.450+0000 7efc59ffb700 1 -- 172.21.15.57:0/2100856668 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4591 (secure 0 0 0) 0x7efc4c015090 con 0x7efc5c06cd40 2023-10-07T23:21:57.457 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:57.458 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/485782907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":485782907},{"type":"v1","addr":"172.21.15.57:6833","nonce":485782907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/112853075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":112853075},{"type":"v1","addr":"172.21.15.57:6835","nonce":112853075}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3600541889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3600541889},{"type":"v1","addr":"172.21.15.154:6835","nonce":3600541889}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:21:25.308405+0000","modified":"2023-10-07T23:21:27.951913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2260010718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2260010718},{"type":"v1","addr":"172.21.15.154:6837","nonce":2260010718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 msgr2=0x7efc48062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 0x7efc48062a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efc50009fd0 tx=0x7efc5000b040 comp rx=0 tx=0).stop 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 msgr2=0x7efc5c154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc5c14fdd0 tx=0x7efc4c017000 comp rx=0 tx=0).stop 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 shutdown_connections 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efc5c156440 0x7efc5c0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efc48060550 0x7efc48062a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc5c0cadd0 0x7efc5c0af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 --2- 172.21.15.57:0/2100856668 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc5c06cd40 0x7efc5c154440 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 >> 172.21.15.57:0/2100856668 conn(0x7efc5c0670e0 msgr2=0x7efc5c0c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 shutdown_connections 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.454+0000 7efc62b5e700 1 -- 172.21.15.57:0/2100856668 wait complete. 2023-10-07T23:21:57.461 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2023-10-07T23:21:57.472 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:21:57.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 Processor -- start 2023-10-07T23:21:57.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 -- start start 2023-10-07T23:21:57.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f4806cf10 0x7f0f481599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4815ccb0 con 0x7f0f480aefb0 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4815cf70 con 0x7f0f480af380 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4cbca700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4815d250 con 0x7f0f4806cf10 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f4806cf10 0x7f0f481599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0f4806cf10 0x7f0f481599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33466/0 (socket says 172.21.15.57:33466) 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 -- 172.21.15.57:0/4264720475 learned_addr learned my addr 172.21.15.57:0/4264720475 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f4659c700 1 --1- 172.21.15.57:0/4264720475 >> v1:172.21.15.57:6789/0 conn(0x7f0f480aefb0 0x7f0f4806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60926/0 (socket says 172.21.15.57:60926) 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 -- 172.21.15.57:0/4264720475 >> v1:172.21.15.154:6789/0 conn(0x7f0f480af380 legacy=0x7f0f48159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 -- 172.21.15.57:0/4264720475 >> v1:172.21.15.57:6789/0 conn(0x7f0f480aefb0 legacy=0x7f0f4806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.578+0000 7f0f45d9b700 1 -- 172.21.15.57:0/4264720475 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4815d530 con 0x7f0f4806cf10 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f45d9b700 1 --2- 172.21.15.57:0/4264720475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f4806cf10 0x7f0f481599e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f20 tx=0x7f0f38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9c5458f05cb955 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4559a700 1 -- 172.21.15.57:0/4264720475 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800e070 con 0x7f0f4806cf10 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4559a700 1 -- 172.21.15.57:0/4264720475 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3800ba80 con 0x7f0f4806cf10 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4559a700 1 -- 172.21.15.57:0/4264720475 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380092e0 con 0x7f0f4806cf10 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- 172.21.15.57:0/4264720475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f4806cf10 msgr2=0x7f0f481599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/4264720475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f4806cf10 0x7f0f481599e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009f20 tx=0x7f0f38009840 comp rx=0 tx=0).stop 2023-10-07T23:21:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- 172.21.15.57:0/4264720475 shutdown_connections 2023-10-07T23:21:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/4264720475 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f4806cf10 0x7f0f481599e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- 172.21.15.57:0/4264720475 >> 172.21.15.57:0/4264720475 conn(0x7f0f48068630 msgr2=0x7f0f48068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- 172.21.15.57:0/4264720475 shutdown_connections 2023-10-07T23:21:57.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- 172.21.15.57:0/4264720475 wait complete. 2023-10-07T23:21:57.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 Processor -- start 2023-10-07T23:21:57.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.582+0000 7f0f4cbca700 1 -- start start 2023-10-07T23:21:57.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f4806cf10 0x7f0f48147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f480aefb0 0x7f0f480b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b3d90 con 0x7f0f480af380 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b4620 con 0x7f0f480aefb0 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480b37d0 con 0x7f0f4806cf10 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f45d9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f480aefb0 0x7f0f480b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4659c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f4806cf10 0x7f0f48147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33478/0 (socket says 172.21.15.57:33478) 2023-10-07T23:21:57.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 -- 172.21.15.57:0/1158111239 learned_addr learned my addr 172.21.15.57:0/1158111239 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 -- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f4806cf10 msgr2=0x7f0f48147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f4806cf10 0x7f0f48147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 -- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f480aefb0 msgr2=0x7f0f480b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f480aefb0 0x7f0f480b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 -- 172.21.15.57:0/1158111239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009ae0 con 0x7f0f480af380 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f46d9d700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c4f0 tx=0x7f0f40007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000ce60 con 0x7f0f480af380 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40011590 con 0x7f0f480af380 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f480b3ab0 con 0x7f0f480af380 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40011930 con 0x7f0f480af380 2023-10-07T23:21:57.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.586+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f480afa00 con 0x7f0f480af380 2023-10-07T23:21:57.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.590+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f480711c0 con 0x7f0f480af380 2023-10-07T23:21:57.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.590+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0f40011ca0 con 0x7f0f480af380 2023-10-07T23:21:57.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.590+0000 7f0f367fc700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 0x7f0f28062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.590+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f4007bab0 con 0x7f0f480af380 2023-10-07T23:21:57.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.590+0000 7f0f4659c700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 0x7f0f28062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.594+0000 7f0f4659c700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 0x7f0f28062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c004fd0 tx=0x7f0f3c007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.598+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0f4004c0a0 con 0x7f0f480af380 2023-10-07T23:21:57.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.758+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f480b4760 con 0x7f0f480af380 2023-10-07T23:21:57.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f367fc700 1 -- 172.21.15.57:0/1158111239 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f0f4004f6c0 con 0x7f0f480af380 2023-10-07T23:21:57.767 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:21:57.767 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":23,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:21:26.784997+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:21:22.957393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-07T23:21:24.292476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:21:57.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 msgr2=0x7f0f28062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 0x7f0f28062a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c004fd0 tx=0x7f0f3c007eb0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 msgr2=0x7f0f4815bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c4f0 tx=0x7f0f40007f50 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 shutdown_connections 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0f480af380 0x7f0f4815bca0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0f28060550 0x7f0f28062a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.762+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0f480aefb0 0x7f0f480b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.766+0000 7f0f4cbca700 1 --2- 172.21.15.57:0/1158111239 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0f4806cf10 0x7f0f48147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.766+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 >> 172.21.15.57:0/1158111239 conn(0x7f0f48068630 msgr2=0x7f0f48156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.766+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 shutdown_connections 2023-10-07T23:21:57.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.766+0000 7f0f4cbca700 1 -- 172.21.15.57:0/1158111239 wait complete. 2023-10-07T23:21:57.782 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T23:21:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 Processor -- start 2023-10-07T23:21:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 -- start start 2023-10-07T23:21:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 0x7f29b006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b0159890 con 0x7f29b006ca30 2023-10-07T23:21:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b0159610 con 0x7f29b00b04f0 2023-10-07T23:21:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b6aed700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00b2b20 con 0x7f29b006e2d0 2023-10-07T23:21:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56712/0 (socket says 172.21.15.57:56712) 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 -- 172.21.15.57:0/2065377743 learned_addr learned my addr 172.21.15.57:0/2065377743 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29affff700 1 --2- 172.21.15.57:0/2065377743 >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 0x7f29b006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 -- 172.21.15.57:0/2065377743 >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 msgr2=0x7f29b006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 --2- 172.21.15.57:0/2065377743 >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 0x7f29b006ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 -- 172.21.15.57:0/2065377743 >> v1:172.21.15.154:6789/0 conn(0x7f29b006ca30 legacy=0x7f29b0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 -- 172.21.15.57:0/2065377743 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00b12c0 con 0x7f29b006e2d0 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29affff700 1 --2- 172.21.15.57:0/2065377743 >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 0x7f29b006ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:21:57.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.886+0000 7f29b508a700 1 --2- 172.21.15.57:0/2065377743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009f20 tx=0x7f29a8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7773edf5da496de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29af7fe700 1 -- 172.21.15.57:0/2065377743 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a800e070 con 0x7f29b006e2d0 2023-10-07T23:21:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29af7fe700 1 -- 172.21.15.57:0/2065377743 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a800b9d0 con 0x7f29b006e2d0 2023-10-07T23:21:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29af7fe700 1 -- 172.21.15.57:0/2065377743 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80092e0 con 0x7f29b006e2d0 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 -- 172.21.15.57:0/2065377743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 msgr2=0x7f29b0068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 --2- 172.21.15.57:0/2065377743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009f20 tx=0x7f29a8009840 comp rx=0 tx=0).stop 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 -- 172.21.15.57:0/2065377743 shutdown_connections 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 --2- 172.21.15.57:0/2065377743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0068fa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 --2- 172.21.15.57:0/2065377743 >> v2:172.21.15.154:3300/0 conn(0x7f29b00b04f0 0x7f29b006ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 -- 172.21.15.57:0/2065377743 >> 172.21.15.57:0/2065377743 conn(0x7f29b0068280 msgr2=0x7f29b006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 -- 172.21.15.57:0/2065377743 shutdown_connections 2023-10-07T23:21:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.890+0000 7f29b6aed700 1 -- 172.21.15.57:0/2065377743 wait complete. 2023-10-07T23:21:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 Processor -- start 2023-10-07T23:21:57.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 -- start start 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 0x7f29b015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015bc30 con 0x7f29b006ca30 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b015c7b0 con 0x7f29b00b04f0 2023-10-07T23:21:57.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b6aed700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b0154760 con 0x7f29b006e2d0 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33484/0 (socket says 172.21.15.57:33484) 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 -- 172.21.15.57:0/49432617 learned_addr learned my addr 172.21.15.57:0/49432617 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29affff700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b508a700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 0x7f29b015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 -- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 msgr2=0x7f29b0154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 -- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 msgr2=0x7f29b015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 0x7f29b015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 -- 172.21.15.57:0/49432617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c009420 con 0x7f29b006ca30 2023-10-07T23:21:57.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b508a700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 0x7f29b015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:57.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.894+0000 7f29b4889700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c4e0 tx=0x7f29a400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.898+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4011590 con 0x7f29b006ca30 2023-10-07T23:21:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.898+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4011710 con 0x7f29b006ca30 2023-10-07T23:21:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.898+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8009ae0 con 0x7f29b006ca30 2023-10-07T23:21:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.898+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4011ad0 con 0x7f29b006ca30 2023-10-07T23:21:57.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b015c160 con 0x7f29b006ca30 2023-10-07T23:21:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b00647b0 con 0x7f29b006ca30 2023-10-07T23:21:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f29a401b3b0 con 0x7f29b006ca30 2023-10-07T23:21:57.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29acff9700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 0x7f2994062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f29a407bd40 con 0x7f29b006ca30 2023-10-07T23:21:57.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29affff700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 0x7f2994062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:57.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.902+0000 7f29affff700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 0x7f2994062a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29b015d8e0 tx=0x7f299c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:57.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:57.910+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f29a404c290 con 0x7f29b006ca30 2023-10-07T23:21:58.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.102+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29b006e770 con 0x7f29b006ca30 2023-10-07T23:21:58.108 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-10-07T23:21:58.102+0000 7f9fa093f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:21:58.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.114+0000 7f29acff9700 1 -- 172.21.15.57:0/49432617 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29a4017020 con 0x7f29b006ca30 2023-10-07T23:21:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 msgr2=0x7f2994062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 0x7f2994062a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f29b015d8e0 tx=0x7f299c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 msgr2=0x7f29b00c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c4e0 tx=0x7f29a400c7f0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 shutdown_connections 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2994060580 0x7f2994062a30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b00b04f0 0x7f29b015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f29b006e2d0 0x7f29b0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 --2- 172.21.15.57:0/49432617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29b006ca30 0x7f29b00c6bc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 >> 172.21.15.57:0/49432617 conn(0x7f29b0068280 msgr2=0x7f29b00afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 shutdown_connections 2023-10-07T23:21:58.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.118+0000 7f29b6aed700 1 -- 172.21.15.57:0/49432617 wait complete. 2023-10-07T23:21:58.124 INFO:tasks.ceph.mds.b.smithi154.stderr: -2040> 2023-10-07T23:21:29.512+0000 7fa214b7d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:21:58.124 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T23:21:58.135 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T23:21:58.148 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-07T23:21:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 Processor -- start 2023-10-07T23:21:58.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 -- start start 2023-10-07T23:21:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7f8c1596f0 0x7f7f8c0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c159c60 con 0x7f7f8c155610 2023-10-07T23:21:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c159f40 con 0x7f7f8c1596f0 2023-10-07T23:21:58.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f90ad1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15a220 con 0x7f7f8c0c9580 2023-10-07T23:21:58.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f8b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7f8c155610 0x7f7f8c1559e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60934/0 (socket says 172.21.15.57:60934) 2023-10-07T23:21:58.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f8b7fe700 1 -- 172.21.15.57:0/2265014273 learned_addr learned my addr 172.21.15.57:0/2265014273 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:21:58.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/2265014273 >> v2:172.21.15.57:3300/0 conn(0x7f7f8c1596f0 0x7f7f8c0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f8bfff700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f8affd700 1 -- 172.21.15.57:0/2265014273 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249809205 0 0) 0x7f7f8c159c60 con 0x7f7f8c155610 2023-10-07T23:21:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f8affd700 1 -- 172.21.15.57:0/2265014273 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f78003570 con 0x7f7f8c155610 2023-10-07T23:21:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f82ffd700 1 -- 172.21.15.57:0/2265014273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 msgr2=0x7f7f8c0c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:58.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f82ffd700 1 -- 172.21.15.57:0/2265014273 >> v1:172.21.15.57:6789/0 conn(0x7f7f8c155610 legacy=0x7f7f8c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.246+0000 7f7f82ffd700 1 -- 172.21.15.57:0/2265014273 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c15a500 con 0x7f7f8c1596f0 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0cb080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7f740098e0 tx=0x7f7f740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd8fd13203bea72 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f8affd700 1 -- 172.21.15.57:0/2265014273 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74011630 con 0x7f7f8c1596f0 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f8affd700 1 -- 172.21.15.57:0/2265014273 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f74011790 con 0x7f7f8c1596f0 2023-10-07T23:21:58.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f8affd700 1 -- 172.21.15.57:0/2265014273 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74011b30 con 0x7f7f8c1596f0 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 -- 172.21.15.57:0/2265014273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1596f0 msgr2=0x7f7f8c0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0cb080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7f740098e0 tx=0x7f7f740091a0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 -- 172.21.15.57:0/2265014273 shutdown_connections 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/2265014273 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0cb080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 -- 172.21.15.57:0/2265014273 >> 172.21.15.57:0/2265014273 conn(0x7f7f8c0670e0 msgr2=0x7f7f8c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:58.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 -- 172.21.15.57:0/2265014273 shutdown_connections 2023-10-07T23:21:58.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.250+0000 7f7f90ad1700 1 -- 172.21.15.57:0/2265014273 wait complete. 2023-10-07T23:21:58.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 Processor -- start 2023-10-07T23:21:58.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 -- start start 2023-10-07T23:21:58.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06ad10 con 0x7f7f8c155610 2023-10-07T23:21:58.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06a450 con 0x7f7f8c1596f0 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c06a6c0 con 0x7f7f8c0c9580 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33496/0 (socket says 172.21.15.57:33496) 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 -- 172.21.15.57:0/4183382845 learned_addr learned my addr 172.21.15.57:0/4183382845 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f8b7fe700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f8bfff700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 -- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 msgr2=0x7f7f8c0ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 -- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8c1596f0 msgr2=0x7f7f8c0b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:58.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:58.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 -- 172.21.15.57:0/4183382845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009420 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f82ffd700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009ed0 tx=0x7f7f74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74000d10 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f74011900 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c06a9a0 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.254+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74025420 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0b0fc0 con 0x7f7f8c155610 2023-10-07T23:21:58.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c0711c0 con 0x7f7f8c155610 2023-10-07T23:21:58.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7f74023070 con 0x7f7f8c155610 2023-10-07T23:21:58.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f83fff700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 0x7f7f6c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:58.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f7f7407dbe0 con 0x7f7f8c155610 2023-10-07T23:21:58.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.258+0000 7f7f8b7fe700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 0x7f7f6c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:58.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.262+0000 7f7f8b7fe700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 0x7f7f6c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:58.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.266+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7f7404e180 con 0x7f7f8c155610 2023-10-07T23:21:58.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:58.466+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7f8c0ca9a0 con 0x7f7f8c155610 2023-10-07T23:21:59.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.118+0000 7f7f83fff700 1 -- 172.21.15.57:0/4183382845 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f740517a0 con 0x7f7f8c155610 2023-10-07T23:21:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 msgr2=0x7f7f6c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 0x7f7f6c062a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c009040 comp rx=0 tx=0).stop 2023-10-07T23:21:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 msgr2=0x7f7f8c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009ed0 tx=0x7f7f74017000 comp rx=0 tx=0).stop 2023-10-07T23:21:59.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 shutdown_connections 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f6c060550 0x7f7f6c062a00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f8c1596f0 0x7f7f8c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7f8c155610 0x7f7f8c0aeda0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 --2- 172.21.15.57:0/4183382845 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7f8c0c9580 0x7f7f8c0ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 >> 172.21.15.57:0/4183382845 conn(0x7f7f8c0670e0 msgr2=0x7f7f8c0cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 shutdown_connections 2023-10-07T23:21:59.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.122+0000 7f7f90ad1700 1 -- 172.21.15.57:0/4183382845 wait complete. 2023-10-07T23:21:59.144 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T23:21:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 Processor -- start 2023-10-07T23:21:59.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 -- start start 2023-10-07T23:21:59.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 0x7f5d280ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2815cb50 0x7f5d280c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d281590d0 0x7f5d280ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d280c9320 con 0x7f5d281590d0 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d28067600 con 0x7f5d28158d00 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2d77b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280add10 con 0x7f5d2815cb50 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d281590d0 0x7f5d280ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5d281590d0 0x7f5d280ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33512/0 (socket says 172.21.15.57:33512) 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 -- 172.21.15.57:0/2940894752 learned_addr learned my addr 172.21.15.57:0/2940894752 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d277fe700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2815cb50 0x7f5d280c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d27fff700 1 --2- 172.21.15.57:0/2940894752 >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 0x7f5d280ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 -- 172.21.15.57:0/2940894752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2815cb50 msgr2=0x7f5d280c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2815cb50 0x7f5d280c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 -- 172.21.15.57:0/2940894752 >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 msgr2=0x7f5d280ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 --2- 172.21.15.57:0/2940894752 >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 0x7f5d280ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 -- 172.21.15.57:0/2940894752 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280c8a80 con 0x7f5d281590d0 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d27fff700 1 --2- 172.21.15.57:0/2940894752 >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 0x7f5d280ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:59.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d2cf7a700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d280ca450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d2806c4e0 tx=0x7f5d20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9413aa0addec4e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d26ffd700 1 -- 172.21.15.57:0/2940894752 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2000e070 con 0x7f5d281590d0 2023-10-07T23:21:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d26ffd700 1 -- 172.21.15.57:0/2940894752 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2000ba20 con 0x7f5d281590d0 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.258+0000 7f5d26ffd700 1 -- 172.21.15.57:0/2940894752 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200092e0 con 0x7f5d281590d0 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 -- 172.21.15.57:0/2940894752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 msgr2=0x7f5d280ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d280ca450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d2806c4e0 tx=0x7f5d20009840 comp rx=0 tx=0).stop 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 -- 172.21.15.57:0/2940894752 shutdown_connections 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d280ca450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/2940894752 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d2815cb50 0x7f5d280c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/2940894752 >> v2:172.21.15.154:3300/0 conn(0x7f5d28158d00 0x7f5d280ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 -- 172.21.15.57:0/2940894752 >> 172.21.15.57:0/2940894752 conn(0x7f5d280692f0 msgr2=0x7f5d280696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 -- 172.21.15.57:0/2940894752 shutdown_connections 2023-10-07T23:21:59.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 -- 172.21.15.57:0/2940894752 wait complete. 2023-10-07T23:21:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.262+0000 7f5d2d77b700 1 Processor -- start 2023-10-07T23:21:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- start start 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d28158d00 0x7f5d280b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 0x7f5d28156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280c2290 con 0x7f5d281590d0 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280c2570 con 0x7f5d2815cb50 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d280c2850 con 0x7f5d28158d00 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33518/0 (socket says 172.21.15.57:33518) 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 -- 172.21.15.57:0/628273202 learned_addr learned my addr 172.21.15.57:0/628273202 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d27fff700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d28158d00 0x7f5d280b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2cf7a700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 0x7f5d28156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 -- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d28158d00 msgr2=0x7f5d280b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d28158d00 0x7f5d280b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 -- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 msgr2=0x7f5d28156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 0x7f5d28156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 -- 172.21.15.57:0/628273202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d20009ae0 con 0x7f5d281590d0 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2cf7a700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 0x7f5d28156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d277fe700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e930 tx=0x7f5d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c013680 con 0x7f5d281590d0 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c0137e0 con 0x7f5d281590d0 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d280c2b30 con 0x7f5d281590d0 2023-10-07T23:21:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c013b80 con 0x7f5d281590d0 2023-10-07T23:21:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.266+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d280c3330 con 0x7f5d281590d0 2023-10-07T23:21:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d28158990 con 0x7f5d281590d0 2023-10-07T23:21:59.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5d1c00c460 con 0x7f5d281590d0 2023-10-07T23:21:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d0ffff700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 0x7f5d14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:21:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5d1c07c950 con 0x7f5d281590d0 2023-10-07T23:21:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d27fff700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 0x7f5d14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:21:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.270+0000 7f5d27fff700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 0x7f5d14062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d18006f00 tx=0x7f5d18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:21:59.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.274+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5d28158990 con 0x7f5d281590d0 2023-10-07T23:21:59.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:21:59.438+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d28158990 con 0x7f5d281590d0 2023-10-07T23:22:00.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.130+0000 7f5d0ffff700 1 -- 172.21.15.57:0/628273202 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5d1c050980 con 0x7f5d281590d0 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.130+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 msgr2=0x7f5d14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 0x7f5d14062a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d18006f00 tx=0x7f5d18008040 comp rx=0 tx=0).stop 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 msgr2=0x7f5d28155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e930 tx=0x7f5d1c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 shutdown_connections 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5d14060550 0x7f5d14062a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5d2815cb50 0x7f5d28156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d281590d0 0x7f5d28155ce0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 --2- 172.21.15.57:0/628273202 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5d28158d00 0x7f5d280b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 >> 172.21.15.57:0/628273202 conn(0x7f5d280692f0 msgr2=0x7f5d2806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 shutdown_connections 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.134+0000 7f5d2d77b700 1 -- 172.21.15.57:0/628273202 wait complete. 2023-10-07T23:22:00.139 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-10-07T23:22:00.150 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T23:22:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 Processor -- start 2023-10-07T23:22:00.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- start start 2023-10-07T23:22:00.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe3f81559d0 0x7fe3f8155da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 0x7fe3f80ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80af620 con 0x7fe3f80ccb50 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80aea10 con 0x7fe3f81559d0 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80aecf0 con 0x7fe3f80adbe0 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe3f81559d0 0x7fe3f8155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe3f81559d0 0x7fe3f8155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33534/0 (socket says 172.21.15.57:33534) 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/2715788743 learned_addr learned my addr 172.21.15.57:0/2715788743 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe9ea700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 0x7fe3f80ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/2715788743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 msgr2=0x7fe3f80ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:00.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 0x7fe3f80ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/2715788743 >> v1:172.21.15.154:6789/0 conn(0x7fe3f80ccb50 legacy=0x7fe3f80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/2715788743 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80aefd0 con 0x7fe3f81559d0 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe9ea700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 0x7fe3f80ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fe1e9700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f81559d0 0x7fe3f8155da0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40099b0 tx=0x7fe3f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fdb3e1589de95 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fd1e7700 1 -- 172.21.15.57:0/2715788743 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400b860 con 0x7fe3f81559d0 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fd1e7700 1 -- 172.21.15.57:0/2715788743 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f400b9c0 con 0x7fe3f81559d0 2023-10-07T23:22:00.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe3fd1e7700 1 -- 172.21.15.57:0/2715788743 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400bd60 con 0x7fe3f81559d0 2023-10-07T23:22:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- 172.21.15.57:0/2715788743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f81559d0 msgr2=0x7fe3f8155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f81559d0 0x7fe3f8155da0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40099b0 tx=0x7fe3f4009250 comp rx=0 tx=0).stop 2023-10-07T23:22:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- 172.21.15.57:0/2715788743 shutdown_connections 2023-10-07T23:22:00.260 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80adbe0 0x7fe3f80ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 --2- 172.21.15.57:0/2715788743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f81559d0 0x7fe3f8155da0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.250+0000 7fe40044d700 1 -- 172.21.15.57:0/2715788743 >> 172.21.15.57:0/2715788743 conn(0x7fe3f806b720 msgr2=0x7fe3f806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- 172.21.15.57:0/2715788743 shutdown_connections 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- 172.21.15.57:0/2715788743 wait complete. 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 Processor -- start 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- start start 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80ccb50 0x7fe3f814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 0x7fe3f814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8165590 con 0x7fe3f80adbe0 2023-10-07T23:22:00.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8166090 con 0x7fe3f81559d0 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80c4b30 con 0x7fe3f80ccb50 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe9ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 0x7fe3f814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fd9e8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80ccb50 0x7fe3f814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe9ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 0x7fe3f814cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:60364/0 (socket says 172.21.15.57:60364) 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe9ea700 1 -- 172.21.15.57:0/1410786481 learned_addr learned my addr 172.21.15.57:0/1410786481 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33550/0 (socket says 172.21.15.57:33550) 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80ccb50 msgr2=0x7fe3f814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80ccb50 0x7fe3f814c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 msgr2=0x7fe3f814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:00.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 0x7fe3f814cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 -- 172.21.15.57:0/1410786481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40094f0 con 0x7fe3f80adbe0 2023-10-07T23:22:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3fe1e9700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3f815a9c0 tx=0x7fe3f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:00.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4011dc0 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f401f070 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f8166520 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40266d0 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.254+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8165a40 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.258+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80647b0 con 0x7fe3f80adbe0 2023-10-07T23:22:00.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.258+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe3f401a430 con 0x7fe3f80adbe0 2023-10-07T23:22:00.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.258+0000 7fe3ee7fc700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 0x7fe3dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:00.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.258+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fe3f4021070 con 0x7fe3f80adbe0 2023-10-07T23:22:00.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.262+0000 7fe3fd9e8700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 0x7fe3dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:00.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.262+0000 7fe3fd9e8700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 0x7fe3dc062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b0d40 tx=0x7fe3e8007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:00.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.266+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe3f4052730 con 0x7fe3f80adbe0 2023-10-07T23:22:00.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:00.422+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3f80ac660 con 0x7fe3f80adbe0 2023-10-07T23:22:01.132 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.126+0000 7fe3ee7fc700 1 -- 172.21.15.57:0/1410786481 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe3f4015020 con 0x7fe3f80adbe0 2023-10-07T23:22:01.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 msgr2=0x7fe3dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 0x7fe3dc062a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80b0d40 tx=0x7fe3e8007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 msgr2=0x7fe3f814c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe3f815a9c0 tx=0x7fe3f4017000 comp rx=0 tx=0).stop 2023-10-07T23:22:01.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 shutdown_connections 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe3dc060550 0x7fe3dc062a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe3f81559d0 0x7fe3f814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe3f80ccb50 0x7fe3f814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 --2- 172.21.15.57:0/1410786481 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe3f80adbe0 0x7fe3f814c020 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 >> 172.21.15.57:0/1410786481 conn(0x7fe3f806b720 msgr2=0x7fe3f814deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 shutdown_connections 2023-10-07T23:22:01.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.138+0000 7fe40044d700 1 -- 172.21.15.57:0/1410786481 wait complete. 2023-10-07T23:22:01.144 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-10-07T23:22:01.159 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T23:22:01.159 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-07T23:22:01.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T23:22:01.160 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:22:01.160 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T23:22:01.160 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-07T23:22:01.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T23:22:01.160 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:22:01.160 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T23:22:01.160 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-07T23:22:01.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T23:22:01.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:22:01.161 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T23:22:01.161 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-07T23:22:01.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T23:22:01.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:22:01.173 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T23:22:01.174 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T23:22:01.175 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T23:22:01.175 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T23:22:01.176 INFO:tasks.ceph.mds.b:Started 2023-10-07T23:22:01.177 DEBUG:teuthology.parallel:result is None 2023-10-07T23:22:01.177 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T23:22:01.177 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T23:22:01.178 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T23:22:01.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T23:22:01.179 INFO:tasks.ceph.mds.d:Started 2023-10-07T23:22:01.179 DEBUG:teuthology.parallel:result is None 2023-10-07T23:22:01.222 INFO:tasks.ceph.mds.c:Started 2023-10-07T23:22:01.222 DEBUG:teuthology.parallel:result is None 2023-10-07T23:22:01.223 INFO:tasks.ceph.mds.a:Started 2023-10-07T23:22:01.223 DEBUG:teuthology.parallel:result is None 2023-10-07T23:22:01.224 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-07T23:22:01.255 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-07T23:22:01.255 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-07T23:22:01.299 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-10-07T23:22:01.300 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-10-07T23:22:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 Processor -- start 2023-10-07T23:22:01.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 -- start start 2023-10-07T23:22:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9464158be0 0x7f94640ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f946406b850 0x7f94640d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946406b3c0 con 0x7f9464158fb0 2023-10-07T23:22:01.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946406a7b0 con 0x7f946406b850 2023-10-07T23:22:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946b473700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946406aa90 con 0x7f9464158be0 2023-10-07T23:22:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f946406b850 0x7f94640d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f946406b850 0x7f94640d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33574/0 (socket says 172.21.15.57:33574) 2023-10-07T23:22:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 -- 172.21.15.57:0/1643257210 learned_addr learned my addr 172.21.15.57:0/1643257210 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:01.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946a471700 1 --2- 172.21.15.57:0/1643257210 >> v2:172.21.15.154:3300/0 conn(0x7f9464158be0 0x7f94640ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 -- 172.21.15.57:0/1643257210 >> v2:172.21.15.154:3300/0 conn(0x7f9464158be0 msgr2=0x7f94640ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 --2- 172.21.15.57:0/1643257210 >> v2:172.21.15.154:3300/0 conn(0x7f9464158be0 0x7f94640ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 -- 172.21.15.57:0/1643257210 >> v1:172.21.15.154:6789/0 conn(0x7f9464158fb0 legacy=0x7f94640cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 -- 172.21.15.57:0/1643257210 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946406ad70 con 0x7f946406b850 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946ac72700 1 --2- 172.21.15.57:0/1643257210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f946406b850 0x7f94640d0330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9464069e50 tx=0x7f945c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25e019c0a03056d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.314+0000 7f946946f700 1 -- 172.21.15.57:0/1643257210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00e070 con 0x7f946406b850 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946946f700 1 -- 172.21.15.57:0/1643257210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00ba50 con 0x7f946406b850 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946946f700 1 -- 172.21.15.57:0/1643257210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0092e0 con 0x7f946406b850 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- 172.21.15.57:0/1643257210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f946406b850 msgr2=0x7f94640d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- 172.21.15.57:0/1643257210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f946406b850 0x7f94640d0330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9464069e50 tx=0x7f945c009840 comp rx=0 tx=0).stop 2023-10-07T23:22:01.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- 172.21.15.57:0/1643257210 shutdown_connections 2023-10-07T23:22:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- 172.21.15.57:0/1643257210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f946406b850 0x7f94640d0330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- 172.21.15.57:0/1643257210 >> v2:172.21.15.154:3300/0 conn(0x7f9464158be0 0x7f94640ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- 172.21.15.57:0/1643257210 >> 172.21.15.57:0/1643257210 conn(0x7f9464068520 msgr2=0x7f9464068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- 172.21.15.57:0/1643257210 shutdown_connections 2023-10-07T23:22:01.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- 172.21.15.57:0/1643257210 wait complete. 2023-10-07T23:22:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 Processor -- start 2023-10-07T23:22:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- start start 2023-10-07T23:22:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 0x7f94641520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9464158fb0 0x7f94640cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464152b30 con 0x7f9464158be0 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94641556c0 con 0x7f946406b850 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946b473700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464152610 con 0x7f9464158fb0 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f9469c70700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f9469c70700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33580/0 (socket says 172.21.15.57:33580) 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f9469c70700 1 -- 172.21.15.57:0/215072062 learned_addr learned my addr 172.21.15.57:0/215072062 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946a471700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 0x7f94641520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.318+0000 7f946ac72700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9464158fb0 0x7f94640cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 -- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9464158fb0 msgr2=0x7f94640cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9464158fb0 0x7f94640cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 -- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 msgr2=0x7f94641520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:01.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 0x7f94641520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 -- 172.21.15.57:0/215072062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009ae0 con 0x7f9464158be0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f946a471700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 0x7f94641520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f9469c70700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f945400c7c0 tx=0x7f9454007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454011590 con 0x7f9464158be0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454011710 con 0x7f9464158be0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9464169820 con 0x7f9464158be0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454011ad0 con 0x7f9464158be0 2023-10-07T23:22:01.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9464155070 con 0x7f9464158be0 2023-10-07T23:22:01.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640711c0 con 0x7f9464158be0 2023-10-07T23:22:01.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f945401b3b0 con 0x7f9464158be0 2023-10-07T23:22:01.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 0x7f9448062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:01.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f945407b900 con 0x7f9464158be0 2023-10-07T23:22:01.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.322+0000 7f946a471700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 0x7f9448062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:01.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.326+0000 7f946a471700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 0x7f9448062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9460006fd0 tx=0x7f9460008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:01.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.330+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f945404c1f0 con 0x7f9464158be0 2023-10-07T23:22:01.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:01.502+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9464155800 con 0x7f9464158be0 2023-10-07T23:22:02.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.958+0000 7f945a7fc700 1 -- 172.21.15.57:0/215072062 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9454017020 con 0x7f9464158be0 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 msgr2=0x7f9448062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 0x7f9448062a00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9460006fd0 tx=0x7f9460008040 comp rx=0 tx=0).stop 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 msgr2=0x7f94640cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f945400c7c0 tx=0x7f9454007f50 comp rx=0 tx=0).stop 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 shutdown_connections 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9464158fb0 0x7f94640cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9464158be0 0x7f94640cc290 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9448060550 0x7f9448062a00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 --2- 172.21.15.57:0/215072062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f946406b850 0x7f94641520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 >> 172.21.15.57:0/215072062 conn(0x7f9464068520 msgr2=0x7f94640c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 shutdown_connections 2023-10-07T23:22:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:02.962+0000 7f946b473700 1 -- 172.21.15.57:0/215072062 wait complete. 2023-10-07T23:22:02.968 INFO:teuthology.orchestra.run.smithi057.stderr: removed all blocklist entries 2023-10-07T23:22:02.982 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-07T23:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.086+0000 7f7fca44b700 1 Processor -- start 2023-10-07T23:22:03.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.086+0000 7f7fca44b700 1 -- start start 2023-10-07T23:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc4156da0 0x7f7fc406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fc40cb020 0x7f7fc40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc40cc4e0 con 0x7f7fc406c9a0 2023-10-07T23:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc4157220 con 0x7f7fc40cb020 2023-10-07T23:22:03.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40cb480 con 0x7f7fc4156da0 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fc40cb020 0x7f7fc40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fc40cb020 0x7f7fc40cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33584/0 (socket says 172.21.15.57:33584) 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 -- 172.21.15.57:0/1013670868 learned_addr learned my addr 172.21.15.57:0/1013670868 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc8c48700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc4156da0 0x7f7fc406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 -- 172.21.15.57:0/1013670868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc4156da0 msgr2=0x7f7fc406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc4156da0 0x7f7fc406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 -- 172.21.15.57:0/1013670868 >> v1:172.21.15.154:6789/0 conn(0x7f7fc406c9a0 legacy=0x7f7fc4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 -- 172.21.15.57:0/1013670868 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40ae7d0 con 0x7f7fc40cb020 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fc9c4a700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc40cb020 0x7f7fc40cbfa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009f20 tx=0x7f7fbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ef0c2b4056dfec server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fb7fff700 1 -- 172.21.15.57:0/1013670868 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00e070 con 0x7f7fc40cb020 2023-10-07T23:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fb7fff700 1 -- 172.21.15.57:0/1013670868 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00b9d0 con 0x7f7fc40cb020 2023-10-07T23:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fb7fff700 1 -- 172.21.15.57:0/1013670868 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0092e0 con 0x7f7fc40cb020 2023-10-07T23:22:03.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- 172.21.15.57:0/1013670868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc40cb020 msgr2=0x7f7fc40cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc40cb020 0x7f7fc40cbfa0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009f20 tx=0x7f7fbc009840 comp rx=0 tx=0).stop 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- 172.21.15.57:0/1013670868 shutdown_connections 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc40cb020 0x7f7fc40cbfa0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 --2- 172.21.15.57:0/1013670868 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc4156da0 0x7f7fc406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- 172.21.15.57:0/1013670868 >> 172.21.15.57:0/1013670868 conn(0x7f7fc40692f0 msgr2=0x7f7fc40696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- 172.21.15.57:0/1013670868 shutdown_connections 2023-10-07T23:22:03.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.090+0000 7f7fca44b700 1 -- 172.21.15.57:0/1013670868 wait complete. 2023-10-07T23:22:03.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 Processor -- start 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- start start 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 0x7f7fc40b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4156da0 0x7f7fc40b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40b1510 con 0x7f7fc406c9a0 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4157370 con 0x7f7fc4156da0 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4157eb0 con 0x7f7fc40cb020 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc8c48700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 0x7f7fc40b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33586/0 (socket says 172.21.15.57:33586) 2023-10-07T23:22:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 -- 172.21.15.57:0/2274729955 learned_addr learned my addr 172.21.15.57:0/2274729955 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:03.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9c4a700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4156da0 0x7f7fc40b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 -- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 msgr2=0x7f7fc40b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 0x7f7fc40b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 -- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4156da0 msgr2=0x7f7fc40b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4156da0 0x7f7fc40b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 -- 172.21.15.57:0/2274729955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc009ae0 con 0x7f7fc406c9a0 2023-10-07T23:22:03.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc8c48700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 0x7f7fc40b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fc9449700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c4e0 tx=0x7f7fb800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8011590 con 0x7f7fc406c9a0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8011710 con 0x7f7fc406c9a0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc4157580 con 0x7f7fc406c9a0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8011ad0 con 0x7f7fc406c9a0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.094+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc4157910 con 0x7f7fc406c9a0 2023-10-07T23:22:03.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc40711c0 con 0x7f7fc406c9a0 2023-10-07T23:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7fb801c440 con 0x7f7fc406c9a0 2023-10-07T23:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fb57fa700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 0x7f7fa8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7fb807b7c0 con 0x7f7fc406c9a0 2023-10-07T23:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fc8c48700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 0x7f7fa8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.098+0000 7f7fc8c48700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 0x7f7fa8062a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40cea70 tx=0x7f7fc00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.106+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7fb804c100 con 0x7f7fc406c9a0 2023-10-07T23:22:03.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7fc40cc710 con 0x7f7fc406c9a0 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fb57fa700 1 -- 172.21.15.57:0/2274729955 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f7fb8017090 con 0x7f7fc406c9a0 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stdout: "auth_dump": [ 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.327 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.a", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFl4EJHMBAAHkCAWP6+9FdWp9FusnpX3w==", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.b", 2023-10-07T23:22:03.328 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFldG8xNxAAj8EFNonJE0X5pMyB0izMxQ==", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.c", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlooFlMxAAG+NVHxNzdNg5cRjAtnAOLQ==", 2023-10-07T23:22:03.329 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.d", 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFloYdROhAA1y7ZdlNsVUsuaarTEcolVQ==", 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.330 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.0", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC65yFlqv5KDhAA25InypbQtn4sjojMQ1q4UA==", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.331 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.1", 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC95yFl3NxlFRAAZBqGACWZ4uyN0fXNtNtuYQ==", 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.2", 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDA5yFlYBmuGBAAgcTTcujrs1CIHWZWM1o2VQ==", 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.333 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.3", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDD5yFlwgfQHhAAZMmLKVbUCsBebAOEy1pBcA==", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.4", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDJ5yFld6uuDhAARh1yNxrm4Ub+VTrik7tLXg==", 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.334 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.5", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDM5yFlaPZvHxAAwYp3I5ZpOy5S1nBrajKpew==", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.335 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.6", 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDP5yFlGvyDJRAAOFvXIQkB+6Q0YyaQ9McSYA==", 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.336 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.7", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDS5yFl445yKxAAh/wYkbCQUJy+nmP4S1lsZA==", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.0", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlzElqBBAAvQsOKVpmlkOCrZOBdS/Mng==", 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.337 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.1", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlydslChAAcyKLUVIvuQ8YjyJ+J0Lepw==", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.338 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.2", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlZJ0YBxAAbk4yS+Sl1PXtOnTxzFikDg==", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:22:03.339 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.3", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlS/3JDxAADLDbTFyercFRlld7VfwQEA==", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:22:03.340 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.admin", 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlzLRqIBAASar5Wgxeyp1nifhwLMYkmw==", 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.341 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-mds", 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFleMlMMxAAP/mnit0EFp7GKeqgiQHmAg==", 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-mds" 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-mgr", 2023-10-07T23:22:03.342 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFlFeNMMxAAELSx3eWL/nnIkf+xKUM0Rw==", 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-osd", 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFl3f1MMxAAD5MjJSK6vBbsO32hX8VDUg==", 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-osd" 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.343 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rbd", 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFlrRRNMxAAeuGcHs4NYCR4fL7CeuybDA==", 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFliS5NMxAAgRGcT6Qxea09ML3tDrG6Ng==", 2023-10-07T23:22:03.344 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.bootstrap-rgw", 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDc5yFlU0ZNMxAAkKaoyCYl+QtmT3SseXvlhw==", 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.345 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.x", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlB/VhLRAAld/Nnm46+mosax2vvzWOJQ==", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.y", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlDPSDKhAALtUKYEj0TqzdoaXF8L+f/Q==", 2023-10-07T23:22:03.346 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:22:03.347 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 msgr2=0x7f7fa8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 0x7f7fa8062a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40cea70 tx=0x7f7fc00091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 msgr2=0x7f7fc40ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800c4e0 tx=0x7f7fb800c7f0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 shutdown_connections 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7fa8060550 0x7f7fa8062a00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.322+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fc4156da0 0x7f7fc40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.326+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fc40cb020 0x7f7fc40b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.326+0000 7f7fca44b700 1 --2- 172.21.15.57:0/2274729955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fc406c9a0 0x7f7fc40ce110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.326+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 >> 172.21.15.57:0/2274729955 conn(0x7f7fc40692f0 msgr2=0x7f7fc40afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.326+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 shutdown_connections 2023-10-07T23:22:03.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.326+0000 7f7fca44b700 1 -- 172.21.15.57:0/2274729955 wait complete. 2023-10-07T23:22:03.350 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-07T23:22:03.463 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 Processor -- start 2023-10-07T23:22:03.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 -- start start 2023-10-07T23:22:03.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f616c159200 0x7f616c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c06ba10 con 0x7f616c0caef0 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c06b0e0 con 0x7f616c0cab20 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.458+0000 7f6170b42700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c06aed0 con 0x7f616c159200 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6169d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f616c0caef0 0x7f616c158af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60960/0 (socket says 172.21.15.57:60960) 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6169d9b700 1 -- 172.21.15.57:0/1751582333 learned_addr learned my addr 172.21.15.57:0/1751582333 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 --2- 172.21.15.57:0/1751582333 >> v2:172.21.15.154:3300/0 conn(0x7f616c159200 0x7f616c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234656282 0 0) 0x7f616c06ba10 con 0x7f616c0caef0 2023-10-07T23:22:03.466 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6158003570 con 0x7f616c0caef0 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175686945 0 0) 0x7f6158003570 con 0x7f616c0caef0 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 >> v2:172.21.15.154:3300/0 conn(0x7f616c159200 msgr2=0x7f616c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 --2- 172.21.15.57:0/1751582333 >> v2:172.21.15.154:3300/0 conn(0x7f616c159200 0x7f616c15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 >> v1:172.21.15.154:6789/0 conn(0x7f616c0cab20 legacy=0x7f616c06da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c06b3c0 con 0x7f616c0caef0 2023-10-07T23:22:03.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f6154001470 con 0x7f616c0caef0 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 >> v1:172.21.15.57:6789/0 conn(0x7f616c0caef0 legacy=0x7f616c158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6158003a60 0x7f6158003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6158004810 0x7f6158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f615800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c06aed0 con 0x7f61580051c0 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615800c0e0 con 0x7f6158004810 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615800c2a0 con 0x7f6158003a60 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f615800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6169d9b700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6158004810 0x7f6158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616a59c700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6158003a60 0x7f6158003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 -- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6158003a60 msgr2=0x7f6158003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6158003a60 0x7f6158003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 -- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6158004810 msgr2=0x7f6158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6158004810 0x7f6158004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 -- 172.21.15.57:0/1751582333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158003570 con 0x7f61580051c0 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616a59c700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6158003a60 0x7f6158003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6169d9b700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6158004810 0x7f6158004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 -- 172.21.15.57:0/1751582333 reap_dead start 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616ad9d700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f615800ba60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f616400e210 tx=0x7f616400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2475719aa3491a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616400eac0 con 0x7f61580051c0 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616400f040 con 0x7f61580051c0 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f616959a700 1 -- 172.21.15.57:0/1751582333 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616400ee70 con 0x7f61580051c0 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 -- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 msgr2=0x7f615800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f615800ba60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f616400e210 tx=0x7f616400b040 comp rx=0 tx=0).stop 2023-10-07T23:22:03.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 -- 172.21.15.57:0/1751582333 shutdown_connections 2023-10-07T23:22:03.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 --2- 172.21.15.57:0/1751582333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f615800ba60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 -- 172.21.15.57:0/1751582333 >> 172.21.15.57:0/1751582333 conn(0x7f616c0ccef0 msgr2=0x7f616c0c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:03.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 -- 172.21.15.57:0/1751582333 shutdown_connections 2023-10-07T23:22:03.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.462+0000 7f6170b42700 1 -- 172.21.15.57:0/1751582333 wait complete. 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 Processor -- start 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 -- start start 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f616c1694b0 0x7f616c0abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f616c0ac040 0x7f616c0ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c16a450 con 0x7f61580051c0 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0ad4c0 con 0x7f616c0ac040 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6170b42700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c169e90 con 0x7f616c1694b0 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33602/0 (socket says 172.21.15.57:33602) 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 -- 172.21.15.57:0/3792315336 learned_addr learned my addr 172.21.15.57:0/3792315336 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f6169d9b700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f616c1694b0 0x7f616c0abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616ad9d700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f616c0ac040 0x7f616c0ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 -- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f616c1694b0 msgr2=0x7f616c0abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f616c1694b0 0x7f616c0abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 -- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f616c0ac040 msgr2=0x7f616c0ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f616c0ac040 0x7f616c0ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 -- 172.21.15.57:0/3792315336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c16a170 con 0x7f61580051c0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.466+0000 7f616a59c700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e5b0 tx=0x7f615c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00ee80 con 0x7f61580051c0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c010040 con 0x7f61580051c0 2023-10-07T23:22:03.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c0ad150 con 0x7f61580051c0 2023-10-07T23:22:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c011670 con 0x7f61580051c0 2023-10-07T23:22:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c0c4070 con 0x7f61580051c0 2023-10-07T23:22:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c0647b0 con 0x7f61580051c0 2023-10-07T23:22:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f615c011a60 con 0x7f61580051c0 2023-10-07T23:22:03.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 0x7f614c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f615c07b580 con 0x7f61580051c0 2023-10-07T23:22:03.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.470+0000 7f6169d9b700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 0x7f614c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.474+0000 7f6169d9b700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 0x7f614c062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f616c06d3f0 tx=0x7f6154007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.478+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f615c04bf70 con 0x7f61580051c0 2023-10-07T23:22:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.698+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f616c06b6a0 con 0x7f61580051c0 2023-10-07T23:22:03.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.706+0000 7f61627fc700 1 -- 172.21.15.57:0/3792315336 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f615c0190a0 con 0x7f61580051c0 2023-10-07T23:22:03.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 msgr2=0x7f614c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 0x7f614c062a00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f616c06d3f0 tx=0x7f6154007040 comp rx=0 tx=0).stop 2023-10-07T23:22:03.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 msgr2=0x7f616c0c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f615c00e5b0 tx=0x7f615c009040 comp rx=0 tx=0).stop 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 shutdown_connections 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f614c060550 0x7f614c062a00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f616c0ac040 0x7f616c0ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f616c1694b0 0x7f616c0abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 --2- 172.21.15.57:0/3792315336 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f61580051c0 0x7f616c0c3990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 >> 172.21.15.57:0/3792315336 conn(0x7f616c0ccef0 msgr2=0x7f616c15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 shutdown_connections 2023-10-07T23:22:03.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.710+0000 7f6170b42700 1 -- 172.21.15.57:0/3792315336 wait complete. 2023-10-07T23:22:03.727 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-07T23:22:03.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 Processor -- start 2023-10-07T23:22:03.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 -- start start 2023-10-07T23:22:03.844 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 0x7fd5c00c9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd5c00c9ec0 0x7fd5c006d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c006c800 con 0x7fd5c0155b30 2023-10-07T23:22:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c006cac0 con 0x7fd5c00c9ec0 2023-10-07T23:22:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bf59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c006cda0 con 0x7fd5c00b1700 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5be59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5be59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33610/0 (socket says 172.21.15.57:33610) 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bdd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 0x7fd5c00c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5bed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd5c00c9ec0 0x7fd5c006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.838+0000 7fd5be59c700 1 -- 172.21.15.57:0/2918783482 learned_addr learned my addr 172.21.15.57:0/2918783482 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 -- 172.21.15.57:0/2918783482 >> v2:172.21.15.154:3300/0 conn(0x7fd5c00c9ec0 msgr2=0x7fd5c006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/2918783482 >> v2:172.21.15.154:3300/0 conn(0x7fd5c00c9ec0 0x7fd5c006d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 -- 172.21.15.57:0/2918783482 >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 msgr2=0x7fd5c00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/2918783482 >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 -- 172.21.15.57:0/2918783482 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c006d080 con 0x7fd5c00b1700 2023-10-07T23:22:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5be59c700 1 --2- 172.21.15.57:0/2918783482 >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/2918783482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 0x7fd5c00c9980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b1b30 tx=0x7fd5a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b521d45fab770917 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bd59a700 1 -- 172.21.15.57:0/2918783482 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011630 con 0x7fd5c00b1700 2023-10-07T23:22:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bd59a700 1 -- 172.21.15.57:0/2918783482 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a80117b0 con 0x7fd5c00b1700 2023-10-07T23:22:03.851 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bd59a700 1 -- 172.21.15.57:0/2918783482 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011b70 con 0x7fd5c00b1700 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- 172.21.15.57:0/2918783482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 msgr2=0x7fd5c00c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/2918783482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 0x7fd5c00c9980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b1b30 tx=0x7fd5a80091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- 172.21.15.57:0/2918783482 shutdown_connections 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/2918783482 >> v2:172.21.15.154:3300/0 conn(0x7fd5c00c9ec0 0x7fd5c006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/2918783482 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c00b1700 0x7fd5c00c9980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/2918783482 >> v2:172.21.15.57:3300/0 conn(0x7fd5c0155b30 0x7fd5c00b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.852 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- 172.21.15.57:0/2918783482 >> 172.21.15.57:0/2918783482 conn(0x7fd5c00cce10 msgr2=0x7fd5c00add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- 172.21.15.57:0/2918783482 shutdown_connections 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- 172.21.15.57:0/2918783482 wait complete. 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 Processor -- start 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.842+0000 7fd5bf59e700 1 -- start start 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 0x7fd5c0158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c0155b30 0x7fd5c01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c015e7c0 con 0x7fd5c00b1700 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0158df0 con 0x7fd5c00c9ec0 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c016c840 con 0x7fd5c0155b30 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33624/0 (socket says 172.21.15.57:33624) 2023-10-07T23:22:03.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 -- 172.21.15.57:0/3818435843 learned_addr learned my addr 172.21.15.57:0/3818435843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bed9d700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c0155b30 0x7fd5c01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 0x7fd5c0158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 -- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c0155b30 msgr2=0x7fd5c01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c0155b30 0x7fd5c01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 -- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 msgr2=0x7fd5c0158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 0x7fd5c0158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 -- 172.21.15.57:0/3818435843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00094f0 con 0x7fd5c00b1700 2023-10-07T23:22:03.855 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 0x7fd5c0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:03.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5be59c700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000ec50 tx=0x7fd5b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000c850 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000c9b0 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8009420 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b000cd30 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c016ce00 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.846+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c00711c0 con 0x7fd5c00b1700 2023-10-07T23:22:03.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.850+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd5b0013680 con 0x7fd5c00b1700 2023-10-07T23:22:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.850+0000 7fd5b67fc700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 0x7fd5a4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.850+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd5b007cb30 con 0x7fd5c00b1700 2023-10-07T23:22:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.850+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 0x7fd5a4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:03.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.850+0000 7fd5bdd9b700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 0x7fd5a4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b1b30 tx=0x7fd5a8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:03.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:03.858+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd5b004d470 con 0x7fd5c00b1700 2023-10-07T23:22:04.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.074+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd5c006d360 con 0x7fd5c00b1700 2023-10-07T23:22:04.093 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.086+0000 7fd5b67fc700 1 -- 172.21.15.57:0/3818435843 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd5b00170a0 con 0x7fd5c00b1700 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 msgr2=0x7fd5a4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 0x7fd5a4062a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b1b30 tx=0x7fd5a8017000 comp rx=0 tx=0).stop 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 msgr2=0x7fd5c006b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000ec50 tx=0x7fd5b0009250 comp rx=0 tx=0).stop 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 shutdown_connections 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5c0155b30 0x7fd5c01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5a4060550 0x7fd5a4062a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5c00c9ec0 0x7fd5c0158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 --2- 172.21.15.57:0/3818435843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5c00b1700 0x7fd5c006b070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 >> 172.21.15.57:0/3818435843 conn(0x7fd5c00cce10 msgr2=0x7fd5c00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 shutdown_connections 2023-10-07T23:22:04.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.090+0000 7fd5bf59e700 1 -- 172.21.15.57:0/3818435843 wait complete. 2023-10-07T23:22:04.112 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-07T23:22:04.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.210+0000 7f408f59e700 1 Processor -- start 2023-10-07T23:22:04.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.210+0000 7f408f59e700 1 -- start start 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408f59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f409006cf10 0x7f409006b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408f59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900b0480 con 0x7f409006ae40 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40901567a0 con 0x7f4090152f30 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408f59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409006b800 con 0x7f409006cf10 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408dd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f409006ae40 0x7f409006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60968/0 (socket says 172.21.15.57:60968) 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f409006cf10 0x7f409006b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408dd9b700 1 -- 172.21.15.57:0/1026533324 learned_addr learned my addr 172.21.15.57:0/1026533324 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408ed9d700 1 -- 172.21.15.57:0/1026533324 >> v1:172.21.15.154:6789/0 conn(0x7f4090152f30 legacy=0x7f40900afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408ed9d700 1 -- 172.21.15.57:0/1026533324 >> v1:172.21.15.57:6789/0 conn(0x7f409006ae40 legacy=0x7f409006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408ed9d700 1 -- 172.21.15.57:0/1026533324 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900af290 con 0x7f409006cf10 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408ed9d700 1 --2- 172.21.15.57:0/1026533324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006cf10 0x7f409006b2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4088009e60 tx=0x7f4088009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398e0b0a52b64edf server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408d59a700 1 -- 172.21.15.57:0/1026533324 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408800bc30 con 0x7f409006cf10 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408d59a700 1 -- 172.21.15.57:0/1026533324 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408800bd90 con 0x7f409006cf10 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.214+0000 7f408d59a700 1 -- 172.21.15.57:0/1026533324 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880196d0 con 0x7f409006cf10 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- 172.21.15.57:0/1026533324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006cf10 msgr2=0x7f409006b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 --2- 172.21.15.57:0/1026533324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006cf10 0x7f409006b2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4088009e60 tx=0x7f4088009780 comp rx=0 tx=0).stop 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- 172.21.15.57:0/1026533324 shutdown_connections 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 --2- 172.21.15.57:0/1026533324 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006cf10 0x7f409006b2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.222 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- 172.21.15.57:0/1026533324 >> 172.21.15.57:0/1026533324 conn(0x7f40900cce10 msgr2=0x7f40900add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- 172.21.15.57:0/1026533324 shutdown_connections 2023-10-07T23:22:04.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- 172.21.15.57:0/1026533324 wait complete. 2023-10-07T23:22:04.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 Processor -- start 2023-10-07T23:22:04.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- start start 2023-10-07T23:22:04.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f409006cf10 0x7f40900c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 0x7f40900b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900a9530 con 0x7f409006ae40 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40900a8f00 con 0x7f409006cf10 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40900cbce0 con 0x7f4090152f30 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33636/0 (socket says 172.21.15.57:33636) 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408e59c700 1 -- 172.21.15.57:0/3608352150 learned_addr learned my addr 172.21.15.57:0/3608352150 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408dd9b700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f409006cf10 0x7f40900c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.218+0000 7f408ed9d700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 0x7f40900b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 -- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 msgr2=0x7f40900b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 0x7f40900b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 -- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f409006cf10 msgr2=0x7f40900c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f409006cf10 0x7f40900c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 -- 172.21.15.57:0/3608352150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088009a20 con 0x7f409006ae40 2023-10-07T23:22:04.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408ed9d700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 0x7f40900b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:04.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408e59c700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f407800c590 tx=0x7f4078007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078011590 con 0x7f409006ae40 2023-10-07T23:22:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078011730 con 0x7f409006ae40 2023-10-07T23:22:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40900cbfc0 con 0x7f409006ae40 2023-10-07T23:22:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078011af0 con 0x7f409006ae40 2023-10-07T23:22:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40900cb850 con 0x7f409006ae40 2023-10-07T23:22:04.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409006b630 con 0x7f409006ae40 2023-10-07T23:22:04.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.222+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4078012420 con 0x7f409006ae40 2023-10-07T23:22:04.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.226+0000 7f40867fc700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 0x7f4070062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.226+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f407807b700 con 0x7f409006ae40 2023-10-07T23:22:04.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.226+0000 7f408dd9b700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 0x7f4070062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.226+0000 7f408dd9b700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 0x7f4070062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4090157ec0 tx=0x7f4080007eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.234+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4078049be0 con 0x7f409006ae40 2023-10-07T23:22:04.456 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.450+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f40900711c0 con 0x7f409006ae40 2023-10-07T23:22:04.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.458+0000 7f40867fc700 1 -- 172.21.15.57:0/3608352150 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f40780170a0 con 0x7f409006ae40 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 msgr2=0x7f4070062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 0x7f4070062a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4090157ec0 tx=0x7f4080007eb0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 msgr2=0x7f4090144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f407800c590 tx=0x7f4078007f30 comp rx=0 tx=0).stop 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 shutdown_connections 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4090152f30 0x7f40900b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4070060550 0x7f4070062a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f409006cf10 0x7f40900c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 --2- 172.21.15.57:0/3608352150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f409006ae40 0x7f4090144f10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 >> 172.21.15.57:0/3608352150 conn(0x7f40900cce10 msgr2=0x7f4090157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 shutdown_connections 2023-10-07T23:22:04.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.462+0000 7f408f59e700 1 -- 172.21.15.57:0/3608352150 wait complete. 2023-10-07T23:22:04.481 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-07T23:22:04.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.582+0000 7f8ae59d0700 1 Processor -- start 2023-10-07T23:22:04.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.582+0000 7f8ae59d0700 1 -- start start 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ae0158e60 0x7f8ae006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae00caba0 con 0x7f8ae0159230 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae006bb20 con 0x7f8ae006c800 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae006b560 con 0x7f8ae0158e60 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ae0158e60 0x7f8ae006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8ae0158e60 0x7f8ae006abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33652/0 (socket says 172.21.15.57:33652) 2023-10-07T23:22:04.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 -- 172.21.15.57:0/676521369 learned_addr learned my addr 172.21.15.57:0/676521369 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 -- 172.21.15.57:0/676521369 >> v1:172.21.15.154:6789/0 conn(0x7f8ae006c800 legacy=0x7f8ae00cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 -- 172.21.15.57:0/676521369 >> v1:172.21.15.57:6789/0 conn(0x7f8ae0159230 legacy=0x7f8ae00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 -- 172.21.15.57:0/676521369 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae006b840 con 0x7f8ae0158e60 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8adeffd700 1 --2- 172.21.15.57:0/676521369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0158e60 0x7f8ae006abc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00099b0 tx=0x7f8ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6adac88ad80c32d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8addffb700 1 -- 172.21.15.57:0/676521369 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000b860 con 0x7f8ae0158e60 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8addffb700 1 -- 172.21.15.57:0/676521369 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad000b9c0 con 0x7f8ae0158e60 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8addffb700 1 -- 172.21.15.57:0/676521369 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000bd60 con 0x7f8ae0158e60 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- 172.21.15.57:0/676521369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0158e60 msgr2=0x7f8ae006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/676521369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0158e60 0x7f8ae006abc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00099b0 tx=0x7f8ad0009250 comp rx=0 tx=0).stop 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- 172.21.15.57:0/676521369 shutdown_connections 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/676521369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0158e60 0x7f8ae006abc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- 172.21.15.57:0/676521369 >> 172.21.15.57:0/676521369 conn(0x7f8ae0068740 msgr2=0x7f8ae0068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- 172.21.15.57:0/676521369 shutdown_connections 2023-10-07T23:22:04.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.586+0000 7f8ae59d0700 1 -- 172.21.15.57:0/676521369 wait complete. 2023-10-07T23:22:04.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 Processor -- start 2023-10-07T23:22:04.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 -- start start 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ae006c800 0x7f8ae00c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae0158e60 0x7f8ae015e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae016a5c0 con 0x7f8ae0159230 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00ca390 con 0x7f8ae0158e60 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.590+0000 7f8ae59d0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae015e950 con 0x7f8ae006c800 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33668/0 (socket says 172.21.15.57:33668) 2023-10-07T23:22:04.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 -- 172.21.15.57:0/3042115586 learned_addr learned my addr 172.21.15.57:0/3042115586 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adeffd700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ae006c800 0x7f8ae00c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ade7fc700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae0158e60 0x7f8ae015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 -- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ae006c800 msgr2=0x7f8ae00c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ae006c800 0x7f8ae00c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 -- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae0158e60 msgr2=0x7f8ae015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae0158e60 0x7f8ae015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 -- 172.21.15.57:0/3042115586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00094f0 con 0x7f8ae0159230 2023-10-07T23:22:04.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8adf7fe700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c910 tx=0x7f8ad8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800a7a0 con 0x7f8ae0159230 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad800a900 con 0x7f8ae0159230 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae00c99c0 con 0x7f8ae0159230 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800acc0 con 0x7f8ae0159230 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae00c9110 con 0x7f8ae0159230 2023-10-07T23:22:04.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae00c9b50 con 0x7f8ae0159230 2023-10-07T23:22:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.594+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8ad8012630 con 0x7f8ae0159230 2023-10-07T23:22:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.598+0000 7f8ad6ffd700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 0x7f8ac0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.598+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ad807c7b0 con 0x7f8ae0159230 2023-10-07T23:22:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.598+0000 7f8adeffd700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 0x7f8ac0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.598+0000 7f8adeffd700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 0x7f8ac0062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00099b0 tx=0x7f8ad0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.611 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.606+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ad8050b80 con 0x7f8ae0159230 2023-10-07T23:22:04.827 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.822+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8ae00647b0 con 0x7f8ae0159230 2023-10-07T23:22:04.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.830+0000 7f8ad6ffd700 1 -- 172.21.15.57:0/3042115586 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8ad8050710 con 0x7f8ae0159230 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 msgr2=0x7f8ac0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 0x7f8ac0062a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00099b0 tx=0x7f8ad0017000 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 msgr2=0x7f8ae006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c910 tx=0x7f8ad8001710 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 shutdown_connections 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8ae0159230 0x7f8ae006a840 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ac0060550 0x7f8ac0062a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ae0158e60 0x7f8ae015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 --2- 172.21.15.57:0/3042115586 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8ae006c800 0x7f8ae00c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 >> 172.21.15.57:0/3042115586 conn(0x7f8ae0068740 msgr2=0x7f8ae01585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 shutdown_connections 2023-10-07T23:22:04.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.834+0000 7f8ae59d0700 1 -- 172.21.15.57:0/3042115586 wait complete. 2023-10-07T23:22:04.853 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-07T23:22:04.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 Processor -- start 2023-10-07T23:22:04.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 -- start start 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f27380ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2738158fb0 0x7f27380ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f273806b850 0x7f27380cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273806b3c0 con 0x7f2738158fb0 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273806a7b0 con 0x7f273806b850 2023-10-07T23:22:04.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273806aa90 con 0x7f2738158be0 2023-10-07T23:22:04.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2738158fb0 0x7f27380ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2738158fb0 0x7f27380ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33674/0 (socket says 172.21.15.57:33674) 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 -- 172.21.15.57:0/477284896 learned_addr learned my addr 172.21.15.57:0/477284896 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f273659c700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f27380ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2736d9d700 1 --2- 172.21.15.57:0/477284896 >> v2:172.21.15.154:3300/0 conn(0x7f273806b850 0x7f27380cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 -- 172.21.15.57:0/477284896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 msgr2=0x7f27380ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f27380ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 -- 172.21.15.57:0/477284896 >> v2:172.21.15.154:3300/0 conn(0x7f273806b850 msgr2=0x7f27380cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 --2- 172.21.15.57:0/477284896 >> v2:172.21.15.154:3300/0 conn(0x7f273806b850 0x7f27380cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.954+0000 7f2735d9b700 1 -- 172.21.15.57:0/477284896 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273806ad70 con 0x7f2738158fb0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f2735d9b700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f27380ce6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f27380c6d40 tx=0x7f272c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc826e7bbc4138fe server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273559a700 1 -- 172.21.15.57:0/477284896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c009e70 con 0x7f2738158fb0 2023-10-07T23:22:04.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273559a700 1 -- 172.21.15.57:0/477284896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c00b860 con 0x7f2738158fb0 2023-10-07T23:22:04.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273559a700 1 -- 172.21.15.57:0/477284896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c00bc00 con 0x7f2738158fb0 2023-10-07T23:22:04.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- 172.21.15.57:0/477284896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 msgr2=0x7f27380ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f27380ce6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f27380c6d40 tx=0x7f272c009250 comp rx=0 tx=0).stop 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- 172.21.15.57:0/477284896 shutdown_connections 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- 172.21.15.57:0/477284896 >> v2:172.21.15.154:3300/0 conn(0x7f273806b850 0x7f27380cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f27380ce6b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- 172.21.15.57:0/477284896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f27380ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- 172.21.15.57:0/477284896 >> 172.21.15.57:0/477284896 conn(0x7f2738068520 msgr2=0x7f2738068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- 172.21.15.57:0/477284896 shutdown_connections 2023-10-07T23:22:04.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- 172.21.15.57:0/477284896 wait complete. 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 Processor -- start 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- start start 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f273806b850 0x7f2738155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f273815ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2738159c60 con 0x7f2738158fb0 2023-10-07T23:22:04.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273815a1e0 con 0x7f273806b850 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f273759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2738155ac0 con 0x7f2738158be0 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f2736d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f2736d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33686/0 (socket says 172.21.15.57:33686) 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.958+0000 7f2736d9d700 1 -- 172.21.15.57:0/2334896690 learned_addr learned my addr 172.21.15.57:0/2334896690 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f273659c700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f273806b850 0x7f2738155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2735d9b700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f273815ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 -- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 msgr2=0x7f273815ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f273815ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 -- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f273806b850 msgr2=0x7f2738155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f273806b850 0x7f2738155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 -- 172.21.15.57:0/2334896690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0094f0 con 0x7f2738158fb0 2023-10-07T23:22:04.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f2736d9d700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2730009fd0 tx=0x7f2730009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000ed50 con 0x7f2738158fb0 2023-10-07T23:22:04.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000eeb0 con 0x7f2738158fb0 2023-10-07T23:22:04.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273816cea0 con 0x7f2738158fb0 2023-10-07T23:22:04.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730014650 con 0x7f2738158fb0 2023-10-07T23:22:04.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2738159e70 con 0x7f2738158fb0 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f27300149c0 con 0x7f2738158fb0 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273815a390 con 0x7f2738158fb0 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 0x7f2718062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f273007cc50 con 0x7f2738158fb0 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.962+0000 7f273659c700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 0x7f2718062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:04.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.966+0000 7f273659c700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 0x7f2718062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2728009d40 tx=0x7f2728008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:04.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:04.970+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f273815a390 con 0x7f2738158fb0 2023-10-07T23:22:05.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.190+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f273815a390 con 0x7f2738158fb0 2023-10-07T23:22:05.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.198+0000 7f27267fc700 1 -- 172.21.15.57:0/2334896690 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f273815a390 con 0x7f2738158fb0 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 msgr2=0x7f2718062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 0x7f2718062a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2728009d40 tx=0x7f2728008040 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 msgr2=0x7f273816c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2730009fd0 tx=0x7f2730009900 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 shutdown_connections 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2738158fb0 0x7f273816c450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2738158be0 0x7f273815ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2718060550 0x7f2718062a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 --2- 172.21.15.57:0/2334896690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f273806b850 0x7f2738155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 >> 172.21.15.57:0/2334896690 conn(0x7f2738068520 msgr2=0x7f273815bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:05.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 shutdown_connections 2023-10-07T23:22:05.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.202+0000 7f273759e700 1 -- 172.21.15.57:0/2334896690 wait complete. 2023-10-07T23:22:05.223 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-07T23:22:05.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 Processor -- start 2023-10-07T23:22:05.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 -- start start 2023-10-07T23:22:05.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd3b8156f00 0x7fd3b80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b80c78a0 con 0x7fd3b8156b30 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b80acef0 con 0x7fd3b8156f00 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3be109700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8068f80 con 0x7fd3b80cd500 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3bc906700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd3b8156f00 0x7fd3b80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3bc906700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd3b8156f00 0x7fd3b80c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33690/0 (socket says 172.21.15.57:33690) 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.326+0000 7fd3bc906700 1 -- 172.21.15.57:0/508647851 learned_addr learned my addr 172.21.15.57:0/508647851 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:05.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bd908700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bc906700 1 -- 172.21.15.57:0/508647851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 msgr2=0x7fd3b806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bc906700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bc906700 1 -- 172.21.15.57:0/508647851 >> v1:172.21.15.154:6789/0 conn(0x7fd3b8156b30 legacy=0x7fd3b80ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bc906700 1 -- 172.21.15.57:0/508647851 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b80c7c90 con 0x7fd3b8156f00 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3bc906700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b80c7210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00099b0 tx=0x7fd3a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d941ad1168dd1f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3b7fff700 1 -- 172.21.15.57:0/508647851 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0009ea0 con 0x7fd3b8156f00 2023-10-07T23:22:05.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3b7fff700 1 -- 172.21.15.57:0/508647851 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a000b860 con 0x7fd3b8156f00 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3b7fff700 1 -- 172.21.15.57:0/508647851 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000bc00 con 0x7fd3b8156f00 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 -- 172.21.15.57:0/508647851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 msgr2=0x7fd3b80c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b80c7210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00099b0 tx=0x7fd3a0009250 comp rx=0 tx=0).stop 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 -- 172.21.15.57:0/508647851 shutdown_connections 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 --2- 172.21.15.57:0/508647851 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b80c7210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 -- 172.21.15.57:0/508647851 >> 172.21.15.57:0/508647851 conn(0x7fd3b806d910 msgr2=0x7fd3b806dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 -- 172.21.15.57:0/508647851 shutdown_connections 2023-10-07T23:22:05.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.330+0000 7fd3be109700 1 -- 172.21.15.57:0/508647851 wait complete. 2023-10-07T23:22:05.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 Processor -- start 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- start start 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b814e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 0x7fd3b80c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b80c64c0 con 0x7fd3b8156f00 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b815c8a0 con 0x7fd3b8156b30 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b80c51c0 con 0x7fd3b80cd500 2023-10-07T23:22:05.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bd908700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bd908700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33696/0 (socket says 172.21.15.57:33696) 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bd908700 1 -- 172.21.15.57:0/1107665766 learned_addr learned my addr 172.21.15.57:0/1107665766 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 0x7fd3b80c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bd107700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b814e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 -- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 msgr2=0x7fd3b814e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b814e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 -- 172.21.15.57:0/1107665766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 msgr2=0x7fd3b815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 -- 172.21.15.57:0/1107665766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a00094f0 con 0x7fd3b8156b30 2023-10-07T23:22:05.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bd908700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3bc906700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 0x7fd3b80c5f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0013000 tx=0x7fd3a0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000bbe0 con 0x7fd3b8156b30 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a001ce30 con 0x7fd3b8156b30 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b80c54a0 con 0x7fd3b8156b30 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a001a6c0 con 0x7fd3b8156b30 2023-10-07T23:22:05.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.334+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b80c5a60 con 0x7fd3b8156b30 2023-10-07T23:22:05.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b80c5c50 con 0x7fd3b8156b30 2023-10-07T23:22:05.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd3a001aaf0 con 0x7fd3b8156b30 2023-10-07T23:22:05.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3b57fa700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 0x7fd398062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd3a007dc90 con 0x7fd3b8156b30 2023-10-07T23:22:05.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3bd107700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 0x7fd398062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.345 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.338+0000 7fd3bd107700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 0x7fd398062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8007fd0 tx=0x7fd3a8000f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.350+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd3a004e5e0 con 0x7fd3b8156b30 2023-10-07T23:22:05.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.566+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd3b80711c0 con 0x7fd3b8156b30 2023-10-07T23:22:05.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3b57fa700 1 -- 172.21.15.57:0/1107665766 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd3a0051c00 con 0x7fd3b8156b30 2023-10-07T23:22:05.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 msgr2=0x7fd398062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 0x7fd398062a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8007fd0 tx=0x7fd3a8000f40 comp rx=0 tx=0).stop 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 msgr2=0x7fd3b80c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 0x7fd3b80c5f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0013000 tx=0x7fd3a0017000 comp rx=0 tx=0).stop 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 shutdown_connections 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3b8156f00 0x7fd3b815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd398060550 0x7fd398062a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b8156b30 0x7fd3b80c5f80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 --2- 172.21.15.57:0/1107665766 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd3b80cd500 0x7fd3b814e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 >> 172.21.15.57:0/1107665766 conn(0x7fd3b806d910 msgr2=0x7fd3b81539d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:05.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.578+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 shutdown_connections 2023-10-07T23:22:05.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.582+0000 7fd3be109700 1 -- 172.21.15.57:0/1107665766 wait complete. 2023-10-07T23:22:05.599 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:22:05.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 Processor -- start 2023-10-07T23:22:05.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 -- start start 2023-10-07T23:22:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a74159130 0x7f8a740c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740c83b0 con 0x7f8a7406bc90 2023-10-07T23:22:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740c8690 con 0x7f8a74158d60 2023-10-07T23:22:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.702+0000 7f8a79f73700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a740c81d0 con 0x7f8a74159130 2023-10-07T23:22:05.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a73fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8a7406bc90 0x7f8a740c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60980/0 (socket says 172.21.15.57:60980) 2023-10-07T23:22:05.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a73fff700 1 -- 172.21.15.57:0/2559372977 learned_addr learned my addr 172.21.15.57:0/2559372977 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:05.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79772700 1 --2- 172.21.15.57:0/2559372977 >> v2:172.21.15.154:3300/0 conn(0x7f8a74159130 0x7f8a740c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a737fe700 1 -- 172.21.15.57:0/2559372977 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492222849 0 0) 0x7f8a740c83b0 con 0x7f8a7406bc90 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a737fe700 1 -- 172.21.15.57:0/2559372977 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a64003570 con 0x7f8a7406bc90 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79772700 1 -- 172.21.15.57:0/2559372977 >> v1:172.21.15.154:6789/0 conn(0x7f8a74158d60 legacy=0x7f8a7406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79772700 1 -- 172.21.15.57:0/2559372977 >> v1:172.21.15.57:6789/0 conn(0x7f8a7406bc90 legacy=0x7f8a740c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79772700 1 -- 172.21.15.57:0/2559372977 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740c8970 con 0x7f8a74159130 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79772700 1 --2- 172.21.15.57:0/2559372977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c7c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e60 tx=0x7f8a6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f170734c5599c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a737fe700 1 -- 172.21.15.57:0/2559372977 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e070 con 0x7f8a74159130 2023-10-07T23:22:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a737fe700 1 -- 172.21.15.57:0/2559372977 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00bb60 con 0x7f8a74159130 2023-10-07T23:22:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a737fe700 1 -- 172.21.15.57:0/2559372977 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c012430 con 0x7f8a74159130 2023-10-07T23:22:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 -- 172.21.15.57:0/2559372977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 msgr2=0x7f8a740c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 --2- 172.21.15.57:0/2559372977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c7c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e60 tx=0x7f8a6c009780 comp rx=0 tx=0).stop 2023-10-07T23:22:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 -- 172.21.15.57:0/2559372977 shutdown_connections 2023-10-07T23:22:05.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 --2- 172.21.15.57:0/2559372977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c7c90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 -- 172.21.15.57:0/2559372977 >> 172.21.15.57:0/2559372977 conn(0x7f8a740670e0 msgr2=0x7f8a740674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:05.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 -- 172.21.15.57:0/2559372977 shutdown_connections 2023-10-07T23:22:05.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.706+0000 7f8a79f73700 1 -- 172.21.15.57:0/2559372977 wait complete. 2023-10-07T23:22:05.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 Processor -- start 2023-10-07T23:22:05.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- start start 2023-10-07T23:22:05.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a7406bc90 0x7f8a7414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7414aac0 con 0x7f8a74158d60 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740c5280 con 0x7f8a74159130 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740afae0 con 0x7f8a7406bc90 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33700/0 (socket says 172.21.15.57:33700) 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79772700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 -- 172.21.15.57:0/793955699 learned_addr learned my addr 172.21.15.57:0/793955699 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:05.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a78f71700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a7406bc90 0x7f8a7414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 -- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a7406bc90 msgr2=0x7f8a7414a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a7406bc90 0x7f8a7414a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 -- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 msgr2=0x7f8a740c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 -- 172.21.15.57:0/793955699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009a20 con 0x7f8a74158d60 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79772700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a73fff700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c790 tx=0x7f8a60007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005c00 con 0x7f8a74158d60 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60005d60 con 0x7f8a74158d60 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a740c3f60 con 0x7f8a74158d60 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600072e0 con 0x7f8a74158d60 2023-10-07T23:22:05.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.710+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a740af1b0 con 0x7f8a74158d60 2023-10-07T23:22:05.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a740c6360 con 0x7f8a74158d60 2023-10-07T23:22:05.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8a60012400 con 0x7f8a74158d60 2023-10-07T23:22:05.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a70ff9700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 0x7f8a58062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:05.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8a6007b7b0 con 0x7f8a74158d60 2023-10-07T23:22:05.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a78f71700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 0x7f8a58062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:05.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.714+0000 7f8a78f71700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 0x7f8a58062a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a68003a80 tx=0x7f8a68007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:05.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.722+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a60049be0 con 0x7f8a74158d60 2023-10-07T23:22:05.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.882+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a740711c0 con 0x7f8a74158d60 2023-10-07T23:22:05.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a70ff9700 1 -- 172.21.15.57:0/793955699 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f8a740c6360 con 0x7f8a74158d60 2023-10-07T23:22:05.890 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:05.890 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:02.953023+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:22:05.892 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 msgr2=0x7f8a58062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 0x7f8a58062a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a68003a80 tx=0x7f8a68007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 msgr2=0x7f8a740c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c790 tx=0x7f8a60007db0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 shutdown_connections 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8a58060550 0x7f8a58062a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a74159130 0x7f8a740c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8a74158d60 0x7f8a740c4d40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 --2- 172.21.15.57:0/793955699 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8a7406bc90 0x7f8a7414a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 >> 172.21.15.57:0/793955699 conn(0x7f8a740670e0 msgr2=0x7f8a741577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:05.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 shutdown_connections 2023-10-07T23:22:05.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:05.886+0000 7f8a79f73700 1 -- 172.21.15.57:0/793955699 wait complete. 2023-10-07T23:22:05.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:22:05.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:22:06.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 Processor -- start 2023-10-07T23:22:06.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 -- start start 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ecc0c6d20 0x7f2ecc0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc0c6890 con 0x7f2ecc0c8400 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0c5ff0 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2d42700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0c8c00 con 0x7f2ecc0c6d20 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33710/0 (socket says 172.21.15.57:33710) 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 -- 172.21.15.57:0/1893944759 learned_addr learned my addr 172.21.15.57:0/1893944759 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:06.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed2541700 1 --2- 172.21.15.57:0/1893944759 >> v2:172.21.15.154:3300/0 conn(0x7f2ecc0c6d20 0x7f2ecc0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 -- 172.21.15.57:0/1893944759 >> v2:172.21.15.154:3300/0 conn(0x7f2ecc0c6d20 msgr2=0x7f2ecc0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 --2- 172.21.15.57:0/1893944759 >> v2:172.21.15.154:3300/0 conn(0x7f2ecc0c6d20 0x7f2ecc0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 -- 172.21.15.57:0/1893944759 >> v1:172.21.15.154:6789/0 conn(0x7f2ecc0c8400 legacy=0x7f2ecc0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 -- 172.21.15.57:0/1893944759 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0ad660 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed153f700 1 --2- 172.21.15.57:0/1893944759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc06d900 tx=0x7f2ec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad594742cb5fb5c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed0d3e700 1 -- 172.21.15.57:0/1893944759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800b860 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed0d3e700 1 -- 172.21.15.57:0/1893944759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800b9c0 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.010+0000 7f2ed0d3e700 1 -- 172.21.15.57:0/1893944759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800bd60 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- 172.21.15.57:0/1893944759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c87d0 msgr2=0x7f2ecc0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/1893944759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc06d900 tx=0x7f2ec8009250 comp rx=0 tx=0).stop 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- 172.21.15.57:0/1893944759 shutdown_connections 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/1893944759 >> v2:172.21.15.154:3300/0 conn(0x7f2ecc0c6d20 0x7f2ecc0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/1893944759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc0ae860 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- 172.21.15.57:0/1893944759 >> 172.21.15.57:0/1893944759 conn(0x7f2ecc0670e0 msgr2=0x7f2ecc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- 172.21.15.57:0/1893944759 shutdown_connections 2023-10-07T23:22:06.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- 172.21.15.57:0/1893944759 wait complete. 2023-10-07T23:22:06.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 Processor -- start 2023-10-07T23:22:06.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.014+0000 7f2ed2d42700 1 -- start start 2023-10-07T23:22:06.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c6d20 0x7f2ecc15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc1673b0 con 0x7f2ecc0c6d20 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc166520 con 0x7f2ecc0c87d0 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc1667a0 con 0x7f2ecc0c8400 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed1d40700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c6d20 0x7f2ecc15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:56922/0 (socket says 172.21.15.57:56922) 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 -- 172.21.15.57:0/2861691425 learned_addr learned my addr 172.21.15.57:0/2861691425 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2541700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 -- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ecc0c87d0 msgr2=0x7f2ecc149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc149d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 -- 172.21.15.57:0/2861691425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c6d20 msgr2=0x7f2ecc15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c6d20 0x7f2ecc15a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 -- 172.21.15.57:0/2861691425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80094f0 con 0x7f2ecc0c8400 2023-10-07T23:22:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed153f700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc06c620 tx=0x7f2ec8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8011e10 con 0x7f2ecc0c8400 2023-10-07T23:22:06.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc166a80 con 0x7f2ecc0c8400 2023-10-07T23:22:06.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec801a420 con 0x7f2ecc0c8400 2023-10-07T23:22:06.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec801a800 con 0x7f2ecc0c8400 2023-10-07T23:22:06.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.018+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc167040 con 0x7f2ecc0c8400 2023-10-07T23:22:06.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc0711c0 con 0x7f2ecc0c8400 2023-10-07T23:22:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2ec801ac30 con 0x7f2ecc0c8400 2023-10-07T23:22:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ec1ffb700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2ec8080580 con 0x7f2ecc0c8400 2023-10-07T23:22:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ed1d40700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.022+0000 7f2ed1d40700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc003a80 tx=0x7f2ebc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.030+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2ec8050e20 con 0x7f2ecc0c8400 2023-10-07T23:22:06.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.194+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ecc0c6240 con 0x7f2ecc0c8400 2023-10-07T23:22:06.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.194+0000 7f2ec1ffb700 1 -- 172.21.15.57:0/2861691425 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ec8015020 con 0x7f2ecc0c8400 2023-10-07T23:22:06.199 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 msgr2=0x7f2eb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc003a80 tx=0x7f2ebc007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 msgr2=0x7f2ecc0bf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc06c620 tx=0x7f2ec8017000 comp rx=0 tx=0).stop 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 shutdown_connections 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ecc0c87d0 0x7f2ecc149d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ecc0c8400 0x7f2ecc0bf4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 --2- 172.21.15.57:0/2861691425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ecc0c6d20 0x7f2ecc15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 >> 172.21.15.57:0/2861691425 conn(0x7f2ecc0670e0 msgr2=0x7f2ecc157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:06.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 shutdown_connections 2023-10-07T23:22:06.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.198+0000 7f2ed2d42700 1 -- 172.21.15.57:0/2861691425 wait complete. 2023-10-07T23:22:06.218 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T23:22:06.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.318+0000 7f79b880e700 1 Processor -- start 2023-10-07T23:22:06.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.318+0000 7f79b880e700 1 -- start start 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79b40aeea0 0x7f79b40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40c80c0 con 0x7f79b40c7b90 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b40c83a0 con 0x7f79b40cb510 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b40c8680 con 0x7f79b40aeea0 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f79b40c7b90 0x7f79b40cae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:60990/0 (socket says 172.21.15.57:60990) 2023-10-07T23:22:06.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 -- 172.21.15.57:0/3760518792 learned_addr learned my addr 172.21.15.57:0/3760518792 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:06.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/3760518792 >> v2:172.21.15.154:3300/0 conn(0x7f79b40aeea0 0x7f79b40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695925673 0 0) 0x7f79b40c80c0 con 0x7f79b40c7b90 2023-10-07T23:22:06.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003570 con 0x7f79b40c7b90 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826072590 0 0) 0x7f799c003570 con 0x7f79b40c7b90 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 >> v2:172.21.15.154:3300/0 conn(0x7f79b40aeea0 msgr2=0x7f79b40c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 --2- 172.21.15.57:0/3760518792 >> v2:172.21.15.154:3300/0 conn(0x7f79b40aeea0 0x7f79b40c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 >> v1:172.21.15.154:6789/0 conn(0x7f79b40cb510 legacy=0x7f79b415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40c8960 con 0x7f79b40c7b90 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f79a8001470 con 0x7f79b40c7b90 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 >> v1:172.21.15.57:6789/0 conn(0x7f79b40c7b90 legacy=0x7f79b40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f799c003a60 0x7f799c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f799c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f799c0051c0 0x7f799c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40c8680 con 0x7f799c004810 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00c0d0 con 0x7f799c0051c0 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c00c290 con 0x7f799c003a60 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f799c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f799c003a60 0x7f799c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b359e700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f799c0051c0 0x7f799c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 -- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f799c003a60 msgr2=0x7f799c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f799c003a60 0x7f799c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 -- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f799c0051c0 msgr2=0x7f799c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f799c0051c0 0x7f799c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 -- 172.21.15.57:0/3760518792 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003570 con 0x7f799c004810 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b359e700 1 -- 172.21.15.57:0/3760518792 reap_dead start 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f799c003a60 0x7f799c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b259c700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f799c004c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c390 tx=0x7f79a8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93a2c27af6bb4b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800cc60 con 0x7f799c004810 2023-10-07T23:22:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800cdc0 con 0x7f799c004810 2023-10-07T23:22:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b1d9b700 1 -- 172.21.15.57:0/3760518792 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8005b50 con 0x7f799c004810 2023-10-07T23:22:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 msgr2=0x7f799c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f799c004c80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c390 tx=0x7f79a8001e50 comp rx=0 tx=0).stop 2023-10-07T23:22:06.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- 172.21.15.57:0/3760518792 shutdown_connections 2023-10-07T23:22:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 --2- 172.21.15.57:0/3760518792 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f799c004c80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- 172.21.15.57:0/3760518792 >> 172.21.15.57:0/3760518792 conn(0x7f79b4095d30 msgr2=0x7f79b406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- 172.21.15.57:0/3760518792 shutdown_connections 2023-10-07T23:22:06.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.322+0000 7f79b880e700 1 -- 172.21.15.57:0/3760518792 wait complete. 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 Processor -- start 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 -- start start 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b40c3b70 0x7f79b40b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b40b1fe0 0x7f79b4148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40c52f0 con 0x7f799c004810 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40c3fc0 con 0x7f79b40c3b70 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.325+0000 7f79b880e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b40c4250 con 0x7f79b40b1fe0 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33742/0 (socket says 172.21.15.57:33742) 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 -- 172.21.15.57:0/2114956425 learned_addr learned my addr 172.21.15.57:0/2114956425 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:06.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b259c700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b40c3b70 0x7f79b40b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b359e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b40b1fe0 0x7f79b4148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 -- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b40b1fe0 msgr2=0x7f79b4148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b40b1fe0 0x7f79b4148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 -- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b40c3b70 msgr2=0x7f79b40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b40c3b70 0x7f79b40b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 -- 172.21.15.57:0/2114956425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b40c4530 con 0x7f799c004810 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b2d9d700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f79a4011700 tx=0x7f79a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4015670 con 0x7f799c004810 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a40157d0 con 0x7f799c004810 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b40c4810 con 0x7f799c004810 2023-10-07T23:22:06.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4015b90 con 0x7f799c004810 2023-10-07T23:22:06.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b40c4dd0 con 0x7f799c004810 2023-10-07T23:22:06.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b40711c0 con 0x7f799c004810 2023-10-07T23:22:06.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.329+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f79a4013410 con 0x7f799c004810 2023-10-07T23:22:06.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.333+0000 7f799affd700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 0x7f79a0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.333+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f79a407cb30 con 0x7f799c004810 2023-10-07T23:22:06.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.333+0000 7f79b259c700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 0x7f79a0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.333+0000 7f79b259c700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 0x7f79a0062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79b40b25b0 tx=0x7f79a8015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.341+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f79a404d470 con 0x7f799c004810 2023-10-07T23:22:06.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79b40c9da0 con 0x7f799c004810 2023-10-07T23:22:06.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f799affd700 1 -- 172.21.15.57:0/2114956425 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f79a4050a90 con 0x7f799c004810 2023-10-07T23:22:06.511 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:06.511 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-07T23:22:06.512 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:06.512 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-10-07T23:22:06.512 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-10-07T23:22:06.512 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:06.512 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-07T23:22:06.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 msgr2=0x7f79a0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 0x7f79a0062a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79b40b25b0 tx=0x7f79a8015000 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 msgr2=0x7f79b415e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f79a4011700 tx=0x7f79a4008040 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 shutdown_connections 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f79b40b1fe0 0x7f79b4148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f79a0060580 0x7f79a0062a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79b40c3b70 0x7f79b40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 --2- 172.21.15.57:0/2114956425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f799c004810 0x7f79b415e490 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.505+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 >> 172.21.15.57:0/2114956425 conn(0x7f79b4095d30 msgr2=0x7f79b40c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.509+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 shutdown_connections 2023-10-07T23:22:06.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.509+0000 7f79b880e700 1 -- 172.21.15.57:0/2114956425 wait complete. 2023-10-07T23:22:06.525 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T23:22:06.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T23:22:06.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.633+0000 7fd2de4b5700 1 Processor -- start 2023-10-07T23:22:06.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.633+0000 7fd2de4b5700 1 -- start start 2023-10-07T23:22:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2de4b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2d80afaf0 0x7fd2d80afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2de4b5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80c9150 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2de4b5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80b0400 con 0x7fd2d8156800 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2de4b5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80c8920 con 0x7fd2d80afaf0 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2ddcb4700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd2d80ad5b0 0x7fd2d80ad9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36186/0 (socket says 172.21.15.57:36186) 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2ddcb4700 1 -- 172.21.15.57:0/219688448 learned_addr learned my addr 172.21.15.57:0/219688448 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/219688448 >> v2:172.21.15.154:3300/0 conn(0x7fd2d80afaf0 0x7fd2d80afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193684731 0 0) 0x7fd2d80c9150 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c4003570 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938454997 0 0) 0x7fd2c4003570 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 >> v2:172.21.15.154:3300/0 conn(0x7fd2d80afaf0 msgr2=0x7fd2d80afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 --2- 172.21.15.57:0/219688448 >> v2:172.21.15.154:3300/0 conn(0x7fd2d80afaf0 0x7fd2d80afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 >> v1:172.21.15.154:6789/0 conn(0x7fd2d8156800 legacy=0x7fd2d8156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80ae300 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7fd2d00017f0 con 0x7fd2d80ad5b0 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 >> v1:172.21.15.57:6789/0 conn(0x7fd2d80ad5b0 legacy=0x7fd2d80ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 0x7fd2c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4004810 0x7fd2c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c40052e0 0x7fd2c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80c8920 con 0x7fd2c4003a60 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c230 con 0x7fd2c4004810 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c3f0 con 0x7fd2c40052e0 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 0x7fd2c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2ddcb4700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c40052e0 0x7fd2c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dccb2700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4004810 0x7fd2c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 -- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c40052e0 msgr2=0x7fd2c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2c40052e0 0x7fd2c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 -- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4004810 msgr2=0x7fd2c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4004810 0x7fd2c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2ddcb4700 1 -- 172.21.15.57:0/219688448 reap_dead start 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 -- 172.21.15.57:0/219688448 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4003570 con 0x7fd2c4003a60 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dccb2700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4004810 0x7fd2c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 0x7fd2c4003e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400bb60 tx=0x7fd2d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a83421d34de9c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400d5d0 con 0x7fd2c4003a60 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d400d730 con 0x7fd2c4003a60 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.637+0000 7fd2cbfff700 1 -- 172.21.15.57:0/219688448 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d400daf0 con 0x7fd2c4003a60 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 msgr2=0x7fd2c4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 0x7fd2c4003e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400bb60 tx=0x7fd2d4008040 comp rx=0 tx=0).stop 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- 172.21.15.57:0/219688448 shutdown_connections 2023-10-07T23:22:06.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/219688448 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2c4003a60 0x7fd2c4003e30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- 172.21.15.57:0/219688448 >> 172.21.15.57:0/219688448 conn(0x7fd2d80692f0 msgr2=0x7fd2d80696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- 172.21.15.57:0/219688448 shutdown_connections 2023-10-07T23:22:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- 172.21.15.57:0/219688448 wait complete. 2023-10-07T23:22:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 Processor -- start 2023-10-07T23:22:06.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.641+0000 7fd2de4b5700 1 -- start start 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4003a60 0x7fd2d816ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d81580e0 0x7fd2d80c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80670e0 con 0x7fd2d8168ec0 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8159540 con 0x7fd2c4003a60 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d816d410 con 0x7fd2d81580e0 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.651 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56654/0 (socket says 172.21.15.57:56654) 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dd4b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4003a60 0x7fd2d816ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 -- 172.21.15.57:0/3721844514 learned_addr learned my addr 172.21.15.57:0/3721844514 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2ddcb4700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d81580e0 0x7fd2d80c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 -- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d81580e0 msgr2=0x7fd2d80c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d81580e0 0x7fd2d80c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 -- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4003a60 msgr2=0x7fd2d816ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4003a60 0x7fd2d816ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 -- 172.21.15.57:0/3721844514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8169480 con 0x7fd2d8168ec0 2023-10-07T23:22:06.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2dccb2700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc000c00 tx=0x7fd2cc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc006820 con 0x7fd2d8168ec0 2023-10-07T23:22:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc006980 con 0x7fd2d8168ec0 2023-10-07T23:22:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d8169760 con 0x7fd2d8168ec0 2023-10-07T23:22:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc006d00 con 0x7fd2d8168ec0 2023-10-07T23:22:06.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.645+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d8169d20 con 0x7fd2d8168ec0 2023-10-07T23:22:06.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d80711c0 con 0x7fd2d8168ec0 2023-10-07T23:22:06.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd2cc007240 con 0x7fd2d8168ec0 2023-10-07T23:22:06.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2c97fa700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 0x7fd2bc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:06.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd2cc07b8f0 con 0x7fd2d8168ec0 2023-10-07T23:22:06.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 0x7fd2bc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:06.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.649+0000 7fd2dd4b3700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 0x7fd2bc062a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400bb60 tx=0x7fd2d401b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:06.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.657+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2cc04c240 con 0x7fd2d8168ec0 2023-10-07T23:22:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:06.817+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd2d80c8b00 con 0x7fd2d8168ec0 2023-10-07T23:22:07.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.165+0000 7fd2c97fa700 1 -- 172.21.15.57:0/3721844514 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd2cc019020 con 0x7fd2d8168ec0 2023-10-07T23:22:07.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 msgr2=0x7fd2bc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 0x7fd2bc062a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400bb60 tx=0x7fd2d401b000 comp rx=0 tx=0).stop 2023-10-07T23:22:07.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 msgr2=0x7fd2d81527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc000c00 tx=0x7fd2cc009040 comp rx=0 tx=0).stop 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 shutdown_connections 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd2d81580e0 0x7fd2d80c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd2d8168ec0 0x7fd2d81527c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd2bc060580 0x7fd2bc062a30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 --2- 172.21.15.57:0/3721844514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2c4003a60 0x7fd2d816ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 >> 172.21.15.57:0/3721844514 conn(0x7fd2d80692f0 msgr2=0x7fd2d80cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 shutdown_connections 2023-10-07T23:22:07.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.177+0000 7fd2de4b5700 1 -- 172.21.15.57:0/3721844514 wait complete. 2023-10-07T23:22:07.185 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-10-07T23:22:07.196 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T23:22:07.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.301+0000 7fe4e5baf700 1 Processor -- start 2023-10-07T23:22:07.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.301+0000 7fe4e5baf700 1 -- start start 2023-10-07T23:22:07.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e0159030 0x7fe4e00aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe4e00ca600 0x7fe4e00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e006a900 con 0x7fe4e006c800 2023-10-07T23:22:07.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00b0450 con 0x7fe4e00ca600 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e006a600 con 0x7fe4e0159030 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe4e00ca600 0x7fe4e00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe4e00ca600 0x7fe4e00caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56664/0 (socket says 172.21.15.57:56664) 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 -- 172.21.15.57:0/1529976275 learned_addr learned my addr 172.21.15.57:0/1529976275 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4df7fe700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e0159030 0x7fe4e00aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 -- 172.21.15.57:0/1529976275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e0159030 msgr2=0x7fe4e00aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e0159030 0x7fe4e00aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 -- 172.21.15.57:0/1529976275 >> v1:172.21.15.154:6789/0 conn(0x7fe4e006c800 legacy=0x7fe4e006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 -- 172.21.15.57:0/1529976275 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00cb350 con 0x7fe4e00ca600 2023-10-07T23:22:07.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4dffff700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e00ca600 0x7fe4e00caa80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009f20 tx=0x7fe4d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503a14cf69718b5f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:07.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4de7fc700 1 -- 172.21.15.57:0/1529976275 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800e070 con 0x7fe4e00ca600 2023-10-07T23:22:07.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4de7fc700 1 -- 172.21.15.57:0/1529976275 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d800ba80 con 0x7fe4e00ca600 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4de7fc700 1 -- 172.21.15.57:0/1529976275 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80092e0 con 0x7fe4e00ca600 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1529976275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e00ca600 msgr2=0x7fe4e00caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e00ca600 0x7fe4e00caa80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009f20 tx=0x7fe4d8009840 comp rx=0 tx=0).stop 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1529976275 shutdown_connections 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e00ca600 0x7fe4e00caa80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1529976275 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e0159030 0x7fe4e00aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1529976275 >> 172.21.15.57:0/1529976275 conn(0x7fe4e0068650 msgr2=0x7fe4e0068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1529976275 shutdown_connections 2023-10-07T23:22:07.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.305+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1529976275 wait complete. 2023-10-07T23:22:07.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 Processor -- start 2023-10-07T23:22:07.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 -- start start 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e00ca600 0x7fe4e0156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4e0159030 0x7fe4e01546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e0157dc0 con 0x7fe4e006c800 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c3fb0 con 0x7fe4e0159030 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4e5baf700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c4270 con 0x7fe4e00ca600 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56680/0 (socket says 172.21.15.57:56680) 2023-10-07T23:22:07.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 -- 172.21.15.57:0/1526691477 learned_addr learned my addr 172.21.15.57:0/1526691477 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4dffff700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4e0159030 0x7fe4e01546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4deffd700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e00ca600 0x7fe4e0156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 -- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e00ca600 msgr2=0x7fe4e0156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e00ca600 0x7fe4e0156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 -- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4e0159030 msgr2=0x7fe4e01546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4e0159030 0x7fe4e01546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 -- 172.21.15.57:0/1526691477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8009ae0 con 0x7fe4e006c800 2023-10-07T23:22:07.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.309+0000 7fe4df7fe700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d860 tx=0x7fe4d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40125f0 con 0x7fe4e006c800 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00b4970 con 0x7fe4e006c800 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4012770 con 0x7fe4e006c800 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4012b30 con 0x7fe4e006c800 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e00b4f30 con 0x7fe4e006c800 2023-10-07T23:22:07.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e00647b0 con 0x7fe4e006c800 2023-10-07T23:22:07.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe4d401e460 con 0x7fe4e006c800 2023-10-07T23:22:07.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 0x7fe4cc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:07.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe4d407cc20 con 0x7fe4e006c800 2023-10-07T23:22:07.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.313+0000 7fe4deffd700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 0x7fe4cc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:07.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.317+0000 7fe4deffd700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 0x7fe4cc062a30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0158370 tx=0x7fe4d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:07.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.325+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe4d4050e80 con 0x7fe4e006c800 2023-10-07T23:22:07.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:07.485+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe4e00cb630 con 0x7fe4e006c800 2023-10-07T23:22:08.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.173+0000 7fe4c77fe700 1 -- 172.21.15.57:0/1526691477 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe4d4050a10 con 0x7fe4e006c800 2023-10-07T23:22:08.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 msgr2=0x7fe4cc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 0x7fe4cc062a30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0158370 tx=0x7fe4d0009040 comp rx=0 tx=0).stop 2023-10-07T23:22:08.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 msgr2=0x7fe4e00b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d860 tx=0x7fe4d4008040 comp rx=0 tx=0).stop 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 shutdown_connections 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe4cc060580 0x7fe4cc062a30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4e0159030 0x7fe4e01546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe4e00ca600 0x7fe4e0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 --2- 172.21.15.57:0/1526691477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4e006c800 0x7fe4e00b4390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 >> 172.21.15.57:0/1526691477 conn(0x7fe4e0068650 msgr2=0x7fe4e00ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 shutdown_connections 2023-10-07T23:22:08.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.193+0000 7fe4e5baf700 1 -- 172.21.15.57:0/1526691477 wait complete. 2023-10-07T23:22:08.201 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-10-07T23:22:08.215 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T23:22:08.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.313+0000 7f2ed53fe700 1 Processor -- start 2023-10-07T23:22:08.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.313+0000 7f2ed53fe700 1 -- start start 2023-10-07T23:22:08.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed0159010 0x7f2ed015d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ed006a6f0 0x7f2ed006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed00ca530 con 0x7f2ed0158c40 2023-10-07T23:22:08.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed006ae60 con 0x7f2ed006a6f0 2023-10-07T23:22:08.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed006b140 con 0x7f2ed0159010 2023-10-07T23:22:08.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ed006a6f0 0x7f2ed006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ed006a6f0 0x7f2ed006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56690/0 (socket says 172.21.15.57:56690) 2023-10-07T23:22:08.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 -- 172.21.15.57:0/2211158103 learned_addr learned my addr 172.21.15.57:0/2211158103 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:08.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ecf7fe700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed0159010 0x7f2ed015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 -- 172.21.15.57:0/2211158103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed0159010 msgr2=0x7f2ed015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed0159010 0x7f2ed015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 -- 172.21.15.57:0/2211158103 >> v1:172.21.15.154:6789/0 conn(0x7f2ed0158c40 legacy=0x7f2ed00afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 -- 172.21.15.57:0/2211158103 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed006b420 con 0x7f2ed006a6f0 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed4bfd700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed006a6f0 0x7f2ed006c800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009f20 tx=0x7f2ec8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58737a03a8d2b27c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2eceffd700 1 -- 172.21.15.57:0/2211158103 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800e070 con 0x7f2ed006a6f0 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2eceffd700 1 -- 172.21.15.57:0/2211158103 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800b9d0 con 0x7f2ed006a6f0 2023-10-07T23:22:08.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2eceffd700 1 -- 172.21.15.57:0/2211158103 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80092e0 con 0x7f2ed006a6f0 2023-10-07T23:22:08.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- 172.21.15.57:0/2211158103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed006a6f0 msgr2=0x7f2ed006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed006a6f0 0x7f2ed006c800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009f20 tx=0x7f2ec8009840 comp rx=0 tx=0).stop 2023-10-07T23:22:08.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- 172.21.15.57:0/2211158103 shutdown_connections 2023-10-07T23:22:08.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed006a6f0 0x7f2ed006c800 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/2211158103 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed0159010 0x7f2ed015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- 172.21.15.57:0/2211158103 >> 172.21.15.57:0/2211158103 conn(0x7f2ed0062ba0 msgr2=0x7f2ed0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:08.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- 172.21.15.57:0/2211158103 shutdown_connections 2023-10-07T23:22:08.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.317+0000 7f2ed53fe700 1 -- 172.21.15.57:0/2211158103 wait complete. 2023-10-07T23:22:08.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 Processor -- start 2023-10-07T23:22:08.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 -- start start 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed006a6f0 0x7f2ed0156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed0159010 0x7f2ed016c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed016ca40 con 0x7f2ed0158c40 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed00b2c70 con 0x7f2ed0159010 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed53fe700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0157880 con 0x7f2ed006a6f0 2023-10-07T23:22:08.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ecf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ecf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56696/0 (socket says 172.21.15.57:56696) 2023-10-07T23:22:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ecf7fe700 1 -- 172.21.15.57:0/1185918243 learned_addr learned my addr 172.21.15.57:0/1185918243 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ed4bfd700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed0159010 0x7f2ed016c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.321+0000 7f2ecffff700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed006a6f0 0x7f2ed0156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 -- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed006a6f0 msgr2=0x7f2ed0156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed006a6f0 0x7f2ed0156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 -- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed0159010 msgr2=0x7f2ed016c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed0159010 0x7f2ed016c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 -- 172.21.15.57:0/1185918243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec8009ae0 con 0x7f2ed0158c40 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ecf7fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800ec80 tx=0x7f2eb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800c850 con 0x7f2ed0158c40 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb800c9b0 con 0x7f2ed0158c40 2023-10-07T23:22:08.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800cd50 con 0x7f2ed0158c40 2023-10-07T23:22:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed016cc70 con 0x7f2ed0158c40 2023-10-07T23:22:08.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.325+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed016d230 con 0x7f2ed0158c40 2023-10-07T23:22:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2eb8013680 con 0x7f2ed0158c40 2023-10-07T23:22:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed00711c0 con 0x7f2ed0158c40 2023-10-07T23:22:08.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ec7fff700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:08.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2eb807c970 con 0x7f2ed0158c40 2023-10-07T23:22:08.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ecffff700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:08.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.329+0000 7f2ecffff700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003a80 tx=0x7f2ec0007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:08.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.337+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2eb804cfc0 con 0x7f2ed0158c40 2023-10-07T23:22:08.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:08.533+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2ed00af620 con 0x7f2ed0158c40 2023-10-07T23:22:09.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.217+0000 7f2ec7fff700 1 -- 172.21.15.57:0/1185918243 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2eb80505e0 con 0x7f2ed0158c40 2023-10-07T23:22:09.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.241+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 msgr2=0x7f2eb0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.241+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0003a80 tx=0x7f2ec0007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.241+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 msgr2=0x7f2ed014c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.241+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800ec80 tx=0x7f2eb80091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.241+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 shutdown_connections 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2eb0060550 0x7f2eb0062a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ed0159010 0x7f2ed016c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ed0158c40 0x7f2ed014c3a0 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800ec80 tx=0x7f2eb80091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 --2- 172.21.15.57:0/1185918243 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ed006a6f0 0x7f2ed0156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 >> 172.21.15.57:0/1185918243 conn(0x7f2ed0062ba0 msgr2=0x7f2ed0158290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:09.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 shutdown_connections 2023-10-07T23:22:09.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.245+0000 7f2ed53fe700 1 -- 172.21.15.57:0/1185918243 wait complete. 2023-10-07T23:22:09.251 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 4 and data pool 5 2023-10-07T23:22:09.261 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:22:09.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 Processor -- start 2023-10-07T23:22:09.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 -- start start 2023-10-07T23:22:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f438014fec0 0x7f43801596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4380150290 0x7f438006b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43800cb2e0 con 0x7f438006d810 2023-10-07T23:22:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4380067590 con 0x7f4380150290 2023-10-07T23:22:09.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.357+0000 7f43869cd700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43800cb5c0 con 0x7f438014fec0 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f4380150290 0x7f438006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43861cc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f438006d810 0x7f438006dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36188/0 (socket says 172.21.15.57:36188) 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43861cc700 1 -- 172.21.15.57:0/723581918 learned_addr learned my addr 172.21.15.57:0/723581918 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43859cb700 1 --2- 172.21.15.57:0/723581918 >> v2:172.21.15.154:3300/0 conn(0x7f438014fec0 0x7f43801596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 -- 172.21.15.57:0/723581918 >> v2:172.21.15.154:3300/0 conn(0x7f438014fec0 msgr2=0x7f43801596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 --2- 172.21.15.57:0/723581918 >> v2:172.21.15.154:3300/0 conn(0x7f438014fec0 0x7f43801596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 -- 172.21.15.57:0/723581918 >> v1:172.21.15.57:6789/0 conn(0x7f438006d810 legacy=0x7f438006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 -- 172.21.15.57:0/723581918 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800cb8a0 con 0x7f4380150290 2023-10-07T23:22:09.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43849c9700 1 -- 172.21.15.57:0/723581918 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610843863 0 0) 0x7f43800cb2e0 con 0x7f438006d810 2023-10-07T23:22:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43851ca700 1 --2- 172.21.15.57:0/723581918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4380150290 0x7f438006b960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f43800c9c60 tx=0x7f43700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531b01675787ba43 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43849c9700 1 -- 172.21.15.57:0/723581918 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011630 con 0x7f4380150290 2023-10-07T23:22:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43849c9700 1 -- 172.21.15.57:0/723581918 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370011790 con 0x7f4380150290 2023-10-07T23:22:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43849c9700 1 -- 172.21.15.57:0/723581918 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370011b30 con 0x7f4380150290 2023-10-07T23:22:09.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 -- 172.21.15.57:0/723581918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4380150290 msgr2=0x7f438006b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 --2- 172.21.15.57:0/723581918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4380150290 0x7f438006b960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f43800c9c60 tx=0x7f43700091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 -- 172.21.15.57:0/723581918 shutdown_connections 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 --2- 172.21.15.57:0/723581918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4380150290 0x7f438006b960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 --2- 172.21.15.57:0/723581918 >> v2:172.21.15.154:3300/0 conn(0x7f438014fec0 0x7f43801596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 -- 172.21.15.57:0/723581918 >> 172.21.15.57:0/723581918 conn(0x7f43800c9750 msgr2=0x7f438006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 -- 172.21.15.57:0/723581918 shutdown_connections 2023-10-07T23:22:09.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.361+0000 7f43869cd700 1 -- 172.21.15.57:0/723581918 wait complete. 2023-10-07T23:22:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 Processor -- start 2023-10-07T23:22:09.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 -- start start 2023-10-07T23:22:09.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438014fec0 0x7f4380156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4380150290 0x7f4380156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800c3a10 con 0x7f438006d810 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800c3c60 con 0x7f4380150290 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43869cd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43800c3f40 con 0x7f438014fec0 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56708/0 (socket says 172.21.15.57:56708) 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 -- 172.21.15.57:0/2786693618 learned_addr learned my addr 172.21.15.57:0/2786693618 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43851ca700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438014fec0 0x7f4380156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43861cc700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4380150290 0x7f4380156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 -- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438014fec0 msgr2=0x7f4380156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438014fec0 0x7f4380156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 -- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4380150290 msgr2=0x7f4380156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4380150290 0x7f4380156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 -- 172.21.15.57:0/2786693618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009420 con 0x7f438006d810 2023-10-07T23:22:09.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f43859cb700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f437c00ec80 tx=0x7f437c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.365+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c00c850 con 0x7f438006d810 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c00c9b0 con 0x7f438006d810 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43800c4220 con 0x7f438006d810 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c00cd50 con 0x7f438006d810 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380153540 con 0x7f438006d810 2023-10-07T23:22:09.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43800711c0 con 0x7f438006d810 2023-10-07T23:22:09.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f437c013680 con 0x7f438006d810 2023-10-07T23:22:09.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f4375ffb700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 0x7f4364062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f437c07c9d0 con 0x7f438006d810 2023-10-07T23:22:09.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.369+0000 7f43851ca700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 0x7f4364062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.373+0000 7f43851ca700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 0x7f4364062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43800c9c60 tx=0x7f4370017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.377+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f437c04cfc0 con 0x7f438006d810 2023-10-07T23:22:09.553 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.549+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43800cbb80 con 0x7f438006d810 2023-10-07T23:22:09.555 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.549+0000 7f4375ffb700 1 -- 172.21.15.57:0/2786693618 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f437c0505e0 con 0x7f438006d810 2023-10-07T23:22:09.556 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:09.556 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":34,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:08.542244+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 msgr2=0x7f4364062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 0x7f4364062a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43800c9c60 tx=0x7f4370017000 comp rx=0 tx=0).stop 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 msgr2=0x7f4380153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f437c00ec80 tx=0x7f437c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 shutdown_connections 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4364060550 0x7f4364062a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.558 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4380150290 0x7f4380156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f438014fec0 0x7f4380156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 --2- 172.21.15.57:0/2786693618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f438006d810 0x7f4380153000 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 >> 172.21.15.57:0/2786693618 conn(0x7f43800c9750 msgr2=0x7f438014e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 shutdown_connections 2023-10-07T23:22:09.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.553+0000 7f43869cd700 1 -- 172.21.15.57:0/2786693618 wait complete. 2023-10-07T23:22:09.570 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:22:09.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 Processor -- start 2023-10-07T23:22:09.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 -- start start 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f725c0cba30 0x7f725c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c157e40 0x7f725c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0ae070 con 0x7f725c0abb30 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0cc340 con 0x7f725c0cba30 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7262ab9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15ada0 con 0x7f725c157e40 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7260855700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f725c0cba30 0x7f725c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7261056700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f725c0abb30 0x7f725c0abf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36204/0 (socket says 172.21.15.57:36204) 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f7261056700 1 -- 172.21.15.57:0/4122526976 learned_addr learned my addr 172.21.15.57:0/4122526976 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:09.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f725bfff700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c157e40 0x7f725c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.669+0000 7f725b7fe700 1 -- 172.21.15.57:0/4122526976 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235481829 0 0) 0x7f725c0ae070 con 0x7f725c0abb30 2023-10-07T23:22:09.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7260855700 1 -- 172.21.15.57:0/4122526976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c157e40 msgr2=0x7f725c0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:09.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7260855700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c157e40 0x7f725c0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7260855700 1 -- 172.21.15.57:0/4122526976 >> v1:172.21.15.57:6789/0 conn(0x7f725c0abb30 legacy=0x7f725c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.679 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7260855700 1 -- 172.21.15.57:0/4122526976 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c15b080 con 0x7f725c0cba30 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7260855700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c0cba30 0x7f725c0c6870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72500098e0 tx=0x7f72500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c17e456b8edb4b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f725b7fe700 1 -- 172.21.15.57:0/4122526976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000b640 con 0x7f725c0cba30 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f725b7fe700 1 -- 172.21.15.57:0/4122526976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725000b7a0 con 0x7f725c0cba30 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f725b7fe700 1 -- 172.21.15.57:0/4122526976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000bb40 con 0x7f725c0cba30 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- 172.21.15.57:0/4122526976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c0cba30 msgr2=0x7f725c0c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c0cba30 0x7f725c0c6870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72500098e0 tx=0x7f72500091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- 172.21.15.57:0/4122526976 shutdown_connections 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c157e40 0x7f725c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 --2- 172.21.15.57:0/4122526976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c0cba30 0x7f725c0c6870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- 172.21.15.57:0/4122526976 >> 172.21.15.57:0/4122526976 conn(0x7f725c0670e0 msgr2=0x7f725c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- 172.21.15.57:0/4122526976 shutdown_connections 2023-10-07T23:22:09.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- 172.21.15.57:0/4122526976 wait complete. 2023-10-07T23:22:09.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 Processor -- start 2023-10-07T23:22:09.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.673+0000 7f7262ab9700 1 -- start start 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c0abb30 0x7f725c16b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 0x7f725c0bea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0aec90 con 0x7f725c157e40 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c16bea0 con 0x7f725c0cba30 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c168ae0 con 0x7f725c0abb30 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56722/0 (socket says 172.21.15.57:56722) 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 -- 172.21.15.57:0/1388428647 learned_addr learned my addr 172.21.15.57:0/1388428647 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7260855700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c0abb30 0x7f725c16b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f725bfff700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 0x7f725c0bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 -- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c0abb30 msgr2=0x7f725c16b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c0abb30 0x7f725c16b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 -- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 msgr2=0x7f725c0bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 0x7f725c0bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 -- 172.21.15.57:0/1388428647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009420 con 0x7f725c157e40 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f725bfff700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 0x7f725c0bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:09.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7261056700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f725400ccc0 tx=0x7f7254001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400a8f0 con 0x7f725c157e40 2023-10-07T23:22:09.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725400aa50 con 0x7f725c157e40 2023-10-07T23:22:09.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c167f60 con 0x7f725c157e40 2023-10-07T23:22:09.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400ae10 con 0x7f725c157e40 2023-10-07T23:22:09.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.677+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c168520 con 0x7f725c157e40 2023-10-07T23:22:09.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c0b0430 con 0x7f725c157e40 2023-10-07T23:22:09.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7254007300 con 0x7f725c157e40 2023-10-07T23:22:09.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7258ff9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 0x7f7248062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f725407cb80 con 0x7f725c157e40 2023-10-07T23:22:09.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7260855700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 0x7f7248062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.681+0000 7f7260855700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 0x7f7248062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72500093f0 tx=0x7f7250017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:09.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.693+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f725c0b0430 con 0x7f725c157e40 2023-10-07T23:22:09.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.857+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f725c0b0430 con 0x7f725c157e40 2023-10-07T23:22:09.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.857+0000 7f7258ff9700 1 -- 172.21.15.57:0/1388428647 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f725c0b0430 con 0x7f725c157e40 2023-10-07T23:22:09.864 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:09.864 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":34,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:08.542244+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:22:09.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 msgr2=0x7f7248062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 0x7f7248062a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72500093f0 tx=0x7f7250017000 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 msgr2=0x7f725c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f725400ccc0 tx=0x7f7254001a80 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 shutdown_connections 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f725c157e40 0x7f725c0abf90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7248060550 0x7f7248062a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f725c0cba30 0x7f725c0bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 --2- 172.21.15.57:0/1388428647 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f725c0abb30 0x7f725c16b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 >> 172.21.15.57:0/1388428647 conn(0x7f725c0670e0 msgr2=0x7f725c15a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 shutdown_connections 2023-10-07T23:22:09.867 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.861+0000 7f7262ab9700 1 -- 172.21.15.57:0/1388428647 wait complete. 2023-10-07T23:22:09.879 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T23:22:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 Processor -- start 2023-10-07T23:22:09.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 -- start start 2023-10-07T23:22:09.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f83f4156b60 0x7f83f4068140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f83f40ac8b0 0x7f83f4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f4068680 con 0x7f83f40acd30 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f415a9b0 con 0x7f83f4156b60 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83fafcc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f41579a0 con 0x7f83f40ac8b0 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83f8d68700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f83f4156b60 0x7f83f4068140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83f8d68700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f83f4156b60 0x7f83f4068140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56724/0 (socket says 172.21.15.57:56724) 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83f8d68700 1 -- 172.21.15.57:0/813823962 learned_addr learned my addr 172.21.15.57:0/813823962 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.989+0000 7f83f3fff700 1 --2- 172.21.15.57:0/813823962 >> v2:172.21.15.154:3300/0 conn(0x7f83f40ac8b0 0x7f83f4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f8d68700 1 -- 172.21.15.57:0/813823962 >> v2:172.21.15.154:3300/0 conn(0x7f83f40ac8b0 msgr2=0x7f83f4156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:09.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f8d68700 1 --2- 172.21.15.57:0/813823962 >> v2:172.21.15.154:3300/0 conn(0x7f83f40ac8b0 0x7f83f4156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:09.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f8d68700 1 -- 172.21.15.57:0/813823962 >> v1:172.21.15.154:6789/0 conn(0x7f83f40acd30 legacy=0x7f83f40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:09.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f8d68700 1 -- 172.21.15.57:0/813823962 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4157c80 con 0x7f83f4156b60 2023-10-07T23:22:09.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f8d68700 1 --2- 172.21.15.57:0/813823962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4068140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83f406b6d0 tx=0x7f83ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc99f98d4f52409 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f37fe700 1 -- 172.21.15.57:0/813823962 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec009e70 con 0x7f83f4156b60 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f37fe700 1 -- 172.21.15.57:0/813823962 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00b860 con 0x7f83f4156b60 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83f37fe700 1 -- 172.21.15.57:0/813823962 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00bc00 con 0x7f83f4156b60 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- 172.21.15.57:0/813823962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 msgr2=0x7f83f4068140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 --2- 172.21.15.57:0/813823962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4068140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83f406b6d0 tx=0x7f83ec009250 comp rx=0 tx=0).stop 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- 172.21.15.57:0/813823962 shutdown_connections 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 --2- 172.21.15.57:0/813823962 >> v2:172.21.15.154:3300/0 conn(0x7f83f40ac8b0 0x7f83f4156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:10.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 --2- 172.21.15.57:0/813823962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4068140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:10.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- 172.21.15.57:0/813823962 >> 172.21.15.57:0/813823962 conn(0x7f83f406b720 msgr2=0x7f83f406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:10.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- 172.21.15.57:0/813823962 shutdown_connections 2023-10-07T23:22:10.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- 172.21.15.57:0/813823962 wait complete. 2023-10-07T23:22:10.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 Processor -- start 2023-10-07T23:22:10.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.993+0000 7f83fafcc700 1 -- start start 2023-10-07T23:22:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f40ac8b0 0x7f83f40a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f40acd30 0x7f83f4169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:10.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40a9f90 con 0x7f83f4156b60 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40aa230 con 0x7f83f40acd30 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40aa510 con 0x7f83f40ac8b0 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56736/0 (socket says 172.21.15.57:56736) 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f40acd30 0x7f83f4169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f8d68700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f40ac8b0 0x7f83f40a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 -- 172.21.15.57:0/1047964245 learned_addr learned my addr 172.21.15.57:0/1047964245 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 -- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f40ac8b0 msgr2=0x7f83f40a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f40ac8b0 0x7f83f40a9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 -- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f40acd30 msgr2=0x7f83f4169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:10.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f40acd30 0x7f83f4169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:10.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 -- 172.21.15.57:0/1047964245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0094f0 con 0x7f83f4156b60 2023-10-07T23:22:10.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f9569700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c560 tx=0x7f83e4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4011590 con 0x7f83f4156b60 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40116f0 con 0x7f83f4156b60 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f40aa7f0 con 0x7f83f4156b60 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4011ab0 con 0x7f83f4156b60 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:09.997+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f40aadb0 con 0x7f83f4156b60 2023-10-07T23:22:10.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f40647b0 con 0x7f83f4156b60 2023-10-07T23:22:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f83e401b3b0 con 0x7f83f4156b60 2023-10-07T23:22:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83f0ff9700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 0x7f83d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:10.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83e407bbf0 con 0x7f83f4156b60 2023-10-07T23:22:10.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83f8d68700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 0x7f83d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:10.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.001+0000 7f83f8d68700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 0x7f83d4062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83f406b6d0 tx=0x7f83ec017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:10.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.009+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f83e404c1e0 con 0x7f83f4156b60 2023-10-07T23:22:10.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:10.209+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83f406d4e0 con 0x7f83f4156b60 2023-10-07T23:22:11.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.217+0000 7f83f0ff9700 1 -- 172.21.15.57:0/1047964245 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83e4017020 con 0x7f83f4156b60 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 msgr2=0x7f83d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 0x7f83d4062a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83f406b6d0 tx=0x7f83ec017000 comp rx=0 tx=0).stop 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 msgr2=0x7f83f4169c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c560 tx=0x7f83e4007ef0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 shutdown_connections 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83f4156b60 0x7f83f4169c40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f83d4060550 0x7f83d4062a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83f40acd30 0x7f83f4169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 --2- 172.21.15.57:0/1047964245 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83f40ac8b0 0x7f83f40a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 >> 172.21.15.57:0/1047964245 conn(0x7f83f406b720 msgr2=0x7f83f40c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:11.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 shutdown_connections 2023-10-07T23:22:11.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.221+0000 7f83fafcc700 1 -- 172.21.15.57:0/1047964245 wait complete. 2023-10-07T23:22:11.247 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T23:22:11.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.361+0000 7f8e94ac4700 1 Processor -- start 2023-10-07T23:22:11.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.361+0000 7f8e94ac4700 1 -- start start 2023-10-07T23:22:11.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e900b1a20 0x7f8e900ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8e900c9ad0 0x7f8e9006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e90156e80 con 0x7f8e90156a00 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9014c860 con 0x7f8e900c9ad0 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e900ad8a0 con 0x7f8e900b1a20 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8e900c9ad0 0x7f8e9006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8e900c9ad0 0x7f8e9006d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56744/0 (socket says 172.21.15.57:56744) 2023-10-07T23:22:11.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 -- 172.21.15.57:0/1764088269 learned_addr learned my addr 172.21.15.57:0/1764088269 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8f7fe700 1 --2- 172.21.15.57:0/1764088269 >> v2:172.21.15.154:3300/0 conn(0x7f8e900b1a20 0x7f8e900ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 -- 172.21.15.57:0/1764088269 >> v2:172.21.15.154:3300/0 conn(0x7f8e900b1a20 msgr2=0x7f8e900ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 --2- 172.21.15.57:0/1764088269 >> v2:172.21.15.154:3300/0 conn(0x7f8e900b1a20 0x7f8e900ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 -- 172.21.15.57:0/1764088269 >> v1:172.21.15.154:6789/0 conn(0x7f8e90156a00 legacy=0x7f8e900c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:11.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 -- 172.21.15.57:0/1764088269 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e90159330 con 0x7f8e900c9ad0 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8ffff700 1 --2- 172.21.15.57:0/1764088269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e900c9ad0 0x7f8e9006d4a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e9006a0e0 tx=0x7f8e88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e420f5f94f69411d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8e7fc700 1 -- 172.21.15.57:0/1764088269 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8800e070 con 0x7f8e900c9ad0 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8e7fc700 1 -- 172.21.15.57:0/1764088269 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8800ba50 con 0x7f8e900c9ad0 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e8e7fc700 1 -- 172.21.15.57:0/1764088269 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880092e0 con 0x7f8e900c9ad0 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- 172.21.15.57:0/1764088269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e900c9ad0 msgr2=0x7f8e9006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:11.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/1764088269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e900c9ad0 0x7f8e9006d4a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8e9006a0e0 tx=0x7f8e88009840 comp rx=0 tx=0).stop 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- 172.21.15.57:0/1764088269 shutdown_connections 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/1764088269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e900c9ad0 0x7f8e9006d4a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/1764088269 >> v2:172.21.15.154:3300/0 conn(0x7f8e900b1a20 0x7f8e900ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- 172.21.15.57:0/1764088269 >> 172.21.15.57:0/1764088269 conn(0x7f8e90067b40 msgr2=0x7f8e900c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- 172.21.15.57:0/1764088269 shutdown_connections 2023-10-07T23:22:11.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.365+0000 7f8e94ac4700 1 -- 172.21.15.57:0/1764088269 wait complete. 2023-10-07T23:22:11.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 Processor -- start 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 -- start start 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 0x7f8e9014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 0x7f8e901516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9014f810 con 0x7f8e90156a00 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e90151210 con 0x7f8e900b1a20 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.369+0000 7f8e94ac4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e90150600 con 0x7f8e900c9ad0 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 0x7f8e901516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8effd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:50870/0 (socket says 172.21.15.57:50870) 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8f7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 0x7f8e9014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8effd700 1 -- 172.21.15.57:0/2410804070 learned_addr learned my addr 172.21.15.57:0/2410804070 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 -- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 msgr2=0x7f8e9014f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 -- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 msgr2=0x7f8e9014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 0x7f8e9014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 -- 172.21.15.57:0/2410804070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e88009ae0 con 0x7f8e90156a00 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8effd700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8f7fe700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 0x7f8e9014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e8ffff700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 0x7f8e901516a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e88015000 tx=0x7f8e88018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8800be60 con 0x7f8e90156a00 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e88008ce0 con 0x7f8e90156a00 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e901508e0 con 0x7f8e90156a00 2023-10-07T23:22:11.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e880266b0 con 0x7f8e90156a00 2023-10-07T23:22:11.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e90150ea0 con 0x7f8e90156a00 2023-10-07T23:22:11.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.373+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e900711c0 con 0x7f8e90156a00 2023-10-07T23:22:11.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.377+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8e88003040 con 0x7f8e90156a00 2023-10-07T23:22:11.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.377+0000 7f8e877fe700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 0x7f8e70062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:11.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.377+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e8807ea60 con 0x7f8e90156a00 2023-10-07T23:22:11.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.377+0000 7f8e8f7fe700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 0x7f8e70062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:11.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.377+0000 7f8e8f7fe700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 0x7f8e70062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006b00 tx=0x7f8e80008400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:11.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.389+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8e8804efb0 con 0x7f8e90156a00 2023-10-07T23:22:11.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:11.585+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8e9006cc50 con 0x7f8e90156a00 2023-10-07T23:22:12.143 INFO:tasks.ceph.mds.c.smithi057.stderr:2023-10-07T23:22:12.137+0000 7f33d9d0c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:22:12.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.221+0000 7f8e877fe700 1 -- 172.21.15.57:0/2410804070 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8e880525d0 con 0x7f8e90156a00 2023-10-07T23:22:12.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 msgr2=0x7f8e70062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 0x7f8e70062a00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8e80006b00 tx=0x7f8e80008400 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 msgr2=0x7f8e901516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 0x7f8e901516a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8e88015000 tx=0x7f8e88018000 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 shutdown_connections 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e90156a00 0x7f8e901516a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8e900c9ad0 0x7f8e9014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8e70060550 0x7f8e70062a00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 --2- 172.21.15.57:0/2410804070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e900b1a20 0x7f8e9014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 >> 172.21.15.57:0/2410804070 conn(0x7f8e90067b40 msgr2=0x7f8e900c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 shutdown_connections 2023-10-07T23:22:12.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.225+0000 7f8e94ac4700 1 -- 172.21.15.57:0/2410804070 wait complete. 2023-10-07T23:22:12.250 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:22:12.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 Processor -- start 2023-10-07T23:22:12.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 -- start start 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f900013f680 0x7f9000069d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90000ca4b0 con 0x7f900013fa50 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90000cadb0 con 0x7f90000afaa0 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f900682c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90000cb090 con 0x7f900013f680 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f9004dc9700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f900013fa50 0x7f90000cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36220/0 (socket says 172.21.15.57:36220) 2023-10-07T23:22:12.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f9004dc9700 1 -- 172.21.15.57:0/3480850745 learned_addr learned my addr 172.21.15.57:0/3480850745 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:12.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffffff700 1 --2- 172.21.15.57:0/3480850745 >> v2:172.21.15.154:3300/0 conn(0x7f900013f680 0x7f9000069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232934292 0 0) 0x7f90000ca4b0 con 0x7f900013fa50 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fec003570 con 0x7f900013fa50 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035012649 0 0) 0x7f90000cadb0 con 0x7f90000afaa0 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90000ca4b0 con 0x7f90000afaa0 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffffff700 1 -- 172.21.15.57:0/3480850745 >> v1:172.21.15.154:6789/0 conn(0x7f90000afaa0 legacy=0x7f90000c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffffff700 1 -- 172.21.15.57:0/3480850745 >> v1:172.21.15.57:6789/0 conn(0x7f900013fa50 legacy=0x7f90000cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffffff700 1 -- 172.21.15.57:0/3480850745 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000cb370 con 0x7f900013f680 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffffff700 1 --2- 172.21.15.57:0/3480850745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013f680 0x7f9000069d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098d0 tx=0x7f8fe8009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10fd251e735d694 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8005c10 con 0x7f900013f680 2023-10-07T23:22:12.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8005d70 con 0x7f900013f680 2023-10-07T23:22:12.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.357+0000 7f8ffeffd700 1 -- 172.21.15.57:0/3480850745 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8006240 con 0x7f900013f680 2023-10-07T23:22:12.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- 172.21.15.57:0/3480850745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013f680 msgr2=0x7f9000069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 --2- 172.21.15.57:0/3480850745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013f680 0x7f9000069d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098d0 tx=0x7f8fe8009120 comp rx=0 tx=0).stop 2023-10-07T23:22:12.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- 172.21.15.57:0/3480850745 shutdown_connections 2023-10-07T23:22:12.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 --2- 172.21.15.57:0/3480850745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013f680 0x7f9000069d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- 172.21.15.57:0/3480850745 >> 172.21.15.57:0/3480850745 conn(0x7f900006d910 msgr2=0x7f900006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:12.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- 172.21.15.57:0/3480850745 shutdown_connections 2023-10-07T23:22:12.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- 172.21.15.57:0/3480850745 wait complete. 2023-10-07T23:22:12.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 Processor -- start 2023-10-07T23:22:12.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.361+0000 7f900682c700 1 -- start start 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900013f680 0x7f9000157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013fa50 0x7f9000158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90001668b0 con 0x7f90000afaa0 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000157ff0 con 0x7f900013fa50 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900016a2f0 con 0x7f900013f680 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56760/0 (socket says 172.21.15.57:56760) 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 -- 172.21.15.57:0/905659318 learned_addr learned my addr 172.21.15.57:0/905659318 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f9004dc9700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013fa50 0x7f9000158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8fff7fe700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900013f680 0x7f9000157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 -- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900013f680 msgr2=0x7f9000157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900013f680 0x7f9000157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 -- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013fa50 msgr2=0x7f9000158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013fa50 0x7f9000158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 -- 172.21.15.57:0/905659318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80093c0 con 0x7f90000afaa0 2023-10-07T23:22:12.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ffffff700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009390 tx=0x7f8fe8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe801d730 con 0x7f90000afaa0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe801d8b0 con 0x7f90000afaa0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900016a5d0 con 0x7f90000afaa0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe801dc70 con 0x7f90000afaa0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.365+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900016ab90 con 0x7f90000afaa0 2023-10-07T23:22:12.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fe801b480 con 0x7f90000afaa0 2023-10-07T23:22:12.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90000711c0 con 0x7f90000afaa0 2023-10-07T23:22:12.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f8ff7fff700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 0x7f8fe0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fe807ea60 con 0x7f90000afaa0 2023-10-07T23:22:12.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f8fff7fe700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 0x7f8fe0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.369+0000 7f8fff7fe700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 0x7f8fe0062a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f900015a5e0 tx=0x7f8ff0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.377+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fe804efb0 con 0x7f90000afaa0 2023-10-07T23:22:12.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9000069660 con 0x7f90000afaa0 2023-10-07T23:22:12.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f8ff7fff700 1 -- 172.21.15.57:0/905659318 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f8fe80525d0 con 0x7f90000afaa0 2023-10-07T23:22:12.583 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:12.583 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:22:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 msgr2=0x7f8fe0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 0x7f8fe0062a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f900015a5e0 tx=0x7f8ff0009040 comp rx=0 tx=0).stop 2023-10-07T23:22:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 msgr2=0x7f9000169d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009390 tx=0x7f8fe8016000 comp rx=0 tx=0).stop 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 shutdown_connections 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fe0060580 0x7f8fe0062a30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f900013fa50 0x7f9000158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f900013f680 0x7f9000157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 --2- 172.21.15.57:0/905659318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90000afaa0 0x7f9000169d10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 >> 172.21.15.57:0/905659318 conn(0x7f900006d910 msgr2=0x7f90000b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.577+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 shutdown_connections 2023-10-07T23:22:12.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.581+0000 7f900682c700 1 -- 172.21.15.57:0/905659318 wait complete. 2023-10-07T23:22:12.587 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:22:12.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:22:12.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.697+0000 7fbdf7fff700 1 Processor -- start 2023-10-07T23:22:12.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.697+0000 7fbdf7fff700 1 -- start start 2023-10-07T23:22:12.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbdf8158d60 0x7fbdf806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdf806bac0 0x7fbdf80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf806be90 con 0x7fbdf8159130 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf80c81d0 con 0x7fbdf806bac0 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf80c83b0 con 0x7fbdf8158d60 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdf806bac0 0x7fbdf80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbdf806bac0 0x7fbdf80c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56762/0 (socket says 172.21.15.57:56762) 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 -- 172.21.15.57:0/1098259088 learned_addr learned my addr 172.21.15.57:0/1098259088 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:12.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf6ffd700 1 --2- 172.21.15.57:0/1098259088 >> v2:172.21.15.154:3300/0 conn(0x7fbdf8158d60 0x7fbdf806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 -- 172.21.15.57:0/1098259088 >> v2:172.21.15.154:3300/0 conn(0x7fbdf8158d60 msgr2=0x7fbdf806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 --2- 172.21.15.57:0/1098259088 >> v2:172.21.15.154:3300/0 conn(0x7fbdf8158d60 0x7fbdf806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 -- 172.21.15.57:0/1098259088 >> v1:172.21.15.154:6789/0 conn(0x7fbdf8159130 legacy=0x7fbdf80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 -- 172.21.15.57:0/1098259088 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf80c8690 con 0x7fbdf806bac0 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf77fe700 1 --2- 172.21.15.57:0/1098259088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c7c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80c9440 tx=0x7fbdf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f8a7b20bb1dfe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf5ffb700 1 -- 172.21.15.57:0/1098259088 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000e070 con 0x7fbdf806bac0 2023-10-07T23:22:12.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf5ffb700 1 -- 172.21.15.57:0/1098259088 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf000ba20 con 0x7fbdf806bac0 2023-10-07T23:22:12.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf5ffb700 1 -- 172.21.15.57:0/1098259088 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00092e0 con 0x7fbdf806bac0 2023-10-07T23:22:12.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- 172.21.15.57:0/1098259088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 msgr2=0x7fbdf80c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/1098259088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c7c90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80c9440 tx=0x7fbdf0009840 comp rx=0 tx=0).stop 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- 172.21.15.57:0/1098259088 shutdown_connections 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/1098259088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c7c90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/1098259088 >> v2:172.21.15.154:3300/0 conn(0x7fbdf8158d60 0x7fbdf806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.701+0000 7fbdf7fff700 1 -- 172.21.15.57:0/1098259088 >> 172.21.15.57:0/1098259088 conn(0x7fbdf80670e0 msgr2=0x7fbdf80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.705+0000 7fbdf7fff700 1 -- 172.21.15.57:0/1098259088 shutdown_connections 2023-10-07T23:22:12.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.705+0000 7fbdf7fff700 1 -- 172.21.15.57:0/1098259088 wait complete. 2023-10-07T23:22:12.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.705+0000 7fbdf7fff700 1 Processor -- start 2023-10-07T23:22:12.712 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.705+0000 7fbdf7fff700 1 -- start start 2023-10-07T23:22:12.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdf8158d60 0x7fbdf80ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdf8159130 0x7fbdf8146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80aeda0 con 0x7fbdf806bac0 2023-10-07T23:22:12.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80afbe0 con 0x7fbdf8158d60 2023-10-07T23:22:12.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf81474b0 con 0x7fbdf8159130 2023-10-07T23:22:12.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56776/0 (socket says 172.21.15.57:56776) 2023-10-07T23:22:12.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 -- 172.21.15.57:0/2587593914 learned_addr learned my addr 172.21.15.57:0/2587593914 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf67fc700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdf8158d60 0x7fbdf80ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf77fe700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdf8159130 0x7fbdf8146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 -- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdf8159130 msgr2=0x7fbdf8146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdf8159130 0x7fbdf8146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 -- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdf8158d60 msgr2=0x7fbdf80ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdf8158d60 0x7fbdf80ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 -- 172.21.15.57:0/2587593914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf0009ae0 con 0x7fbdf806bac0 2023-10-07T23:22:12.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf6ffd700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbde800ec80 tx=0x7fbde80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800c8e0 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde800ca40 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800cde0 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf80aefd0 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf80af590 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf80711c0 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.709+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbde8013680 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.713+0000 7fbddeffd700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 0x7fbde4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.713+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbde807cc00 con 0x7fbdf806bac0 2023-10-07T23:22:12.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.713+0000 7fbdf67fc700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 0x7fbde4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:12.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.713+0000 7fbdf67fc700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 0x7fbde4062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80687c0 tx=0x7fbdec007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:12.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.721+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbde804d150 con 0x7fbdf806bac0 2023-10-07T23:22:12.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.885+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdf80c8970 con 0x7fbdf806bac0 2023-10-07T23:22:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbddeffd700 1 -- 172.21.15.57:0/2587593914 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fbde8050770 con 0x7fbdf806bac0 2023-10-07T23:22:12.895 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:12.896 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:10.194488+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:22:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 msgr2=0x7fbde4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 0x7fbde4062a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbdf80687c0 tx=0x7fbdec007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 msgr2=0x7fbdf80c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbde800ec80 tx=0x7fbde80091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 shutdown_connections 2023-10-07T23:22:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbdf8159130 0x7fbdf8146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbde4060550 0x7fbde4062a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdf8158d60 0x7fbdf80ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 --2- 172.21.15.57:0/2587593914 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbdf806bac0 0x7fbdf80c5510 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:12.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.889+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 >> 172.21.15.57:0/2587593914 conn(0x7fbdf80670e0 msgr2=0x7fbdf80c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:12.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.893+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 shutdown_connections 2023-10-07T23:22:12.900 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:12.893+0000 7fbdf7fff700 1 -- 172.21.15.57:0/2587593914 wait complete. 2023-10-07T23:22:12.910 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T23:22:12.910 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:22:13.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.009+0000 7fa98f59e700 1 Processor -- start 2023-10-07T23:22:13.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.009+0000 7fa98f59e700 1 -- start start 2023-10-07T23:22:13.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa9900ce880 0x7fa9900cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa990158fb0 0x7fa9900d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99006b3c0 con 0x7fa990158be0 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9900d06a0 con 0x7fa990158fb0 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99006a7b0 con 0x7fa9900ce880 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa990158fb0 0x7fa9900d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98e59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa990158be0 0x7fa9900ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36232/0 (socket says 172.21.15.57:36232) 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98e59c700 1 -- 172.21.15.57:0/667691824 learned_addr learned my addr 172.21.15.57:0/667691824 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98dd9b700 1 --2- 172.21.15.57:0/667691824 >> v2:172.21.15.154:3300/0 conn(0x7fa9900ce880 0x7fa9900cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 -- 172.21.15.57:0/667691824 >> v2:172.21.15.154:3300/0 conn(0x7fa9900ce880 msgr2=0x7fa9900cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 --2- 172.21.15.57:0/667691824 >> v2:172.21.15.154:3300/0 conn(0x7fa9900ce880 0x7fa9900cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 -- 172.21.15.57:0/667691824 >> v1:172.21.15.57:6789/0 conn(0x7fa990158be0 legacy=0x7fa9900ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 -- 172.21.15.57:0/667691824 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99006aa90 con 0x7fa990158fb0 2023-10-07T23:22:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98ed9d700 1 --2- 172.21.15.57:0/667691824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa9900d0160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa988009e60 tx=0x7fa988009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97c9577be831cad server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98d59a700 1 -- 172.21.15.57:0/667691824 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98800e070 con 0x7fa990158fb0 2023-10-07T23:22:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98d59a700 1 -- 172.21.15.57:0/667691824 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98800bb20 con 0x7fa990158fb0 2023-10-07T23:22:13.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98d59a700 1 -- 172.21.15.57:0/667691824 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988012430 con 0x7fa990158fb0 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- 172.21.15.57:0/667691824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 msgr2=0x7fa9900d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 --2- 172.21.15.57:0/667691824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa9900d0160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa988009e60 tx=0x7fa988009780 comp rx=0 tx=0).stop 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- 172.21.15.57:0/667691824 shutdown_connections 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 --2- 172.21.15.57:0/667691824 >> v2:172.21.15.154:3300/0 conn(0x7fa9900ce880 0x7fa9900cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 --2- 172.21.15.57:0/667691824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa9900d0160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- 172.21.15.57:0/667691824 >> 172.21.15.57:0/667691824 conn(0x7fa990068520 msgr2=0x7fa990068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- 172.21.15.57:0/667691824 shutdown_connections 2023-10-07T23:22:13.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.013+0000 7fa98f59e700 1 -- 172.21.15.57:0/667691824 wait complete. 2023-10-07T23:22:13.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 Processor -- start 2023-10-07T23:22:13.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 -- start start 2023-10-07T23:22:13.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 0x7fa9900c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99014b710 con 0x7fa990158fb0 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900c74a0 con 0x7fa990158be0 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa990152630 con 0x7fa9900ce880 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:50898/0 (socket says 172.21.15.57:50898) 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98e59c700 1 -- 172.21.15.57:0/96430819 learned_addr learned my addr 172.21.15.57:0/96430819 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 0x7fa9900c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56798/0 (socket says 172.21.15.57:56798) 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 -- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 msgr2=0x7fa990157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 -- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 msgr2=0x7fa9900c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 0x7fa9900c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98ed9d700 1 -- 172.21.15.57:0/96430819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988009a20 con 0x7fa990158fb0 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98e59c700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:13.027 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.017+0000 7fa98dd9b700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 0x7fa9900c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa98ed9d700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa988016000 tx=0x7fa988019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988027470 con 0x7fa990158fb0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988027610 con 0x7fa990158fb0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9900c6e50 con 0x7fa990158fb0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880279d0 con 0x7fa990158fb0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa990152910 con 0x7fa990158fb0 2023-10-07T23:22:13.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9900c72d0 con 0x7fa990158fb0 2023-10-07T23:22:13.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.021+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa988024070 con 0x7fa990158fb0 2023-10-07T23:22:13.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.025+0000 7fa97e7fc700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 0x7fa970062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.025+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa98807ee00 con 0x7fa990158fb0 2023-10-07T23:22:13.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.025+0000 7fa98e59c700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 0x7fa970062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.025+0000 7fa98e59c700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 0x7fa970062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa984000c00 tx=0x7fa984009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.033+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa98804f3f0 con 0x7fa990158fb0 2023-10-07T23:22:13.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.189+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9900c5b00 con 0x7fa970060550 2023-10-07T23:22:13.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.209+0000 7fa97e7fc700 1 -- 172.21.15.57:0/96430819 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+267346 (secure 0 0 0) 0x7fa98400fa90 con 0x7fa970060550 2023-10-07T23:22:13.218 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 msgr2=0x7fa970062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 0x7fa970062a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa984000c00 tx=0x7fa984009040 comp rx=0 tx=0).stop 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 msgr2=0x7fa99014b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa988016000 tx=0x7fa988019000 comp rx=0 tx=0).stop 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 shutdown_connections 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa990158fb0 0x7fa99014b1d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa970060550 0x7fa970062a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa990158be0 0x7fa9900c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 --2- 172.21.15.57:0/96430819 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa9900ce880 0x7fa990157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 >> 172.21.15.57:0/96430819 conn(0x7fa990068520 msgr2=0x7fa9900688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 shutdown_connections 2023-10-07T23:22:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.213+0000 7fa98f59e700 1 -- 172.21.15.57:0/96430819 wait complete. 2023-10-07T23:22:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:22:13.233 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-10-07T23:22:13.086755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398952,"kb_used_data":4264,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575768,"statfs":{"total":773094113280,"available":770637586432,"internally_reserved":0,"allocated":4366336,"data_stored":2377300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.917060"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.230521+0000","last_change":"2023-10-07T23:22:09.227740+0000","last_active":"2023-10-07T23:22:09.230521+0000","last_peered":"2023-10-07T23:22:09.230521+0000","last_clean":"2023-10-07T23:22:09.230521+0000","last_became_active":"2023-10-07T23:22:09.227359+0000","last_became_peered":"2023-10-07T23:22:09.227359+0000","last_unstale":"2023-10-07T23:22:09.230521+0000","last_undegraded":"2023-10-07T23:22:09.230521+0000","last_fullsized":"2023-10-07T23:22:09.230521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247996+0000","last_change":"2023-10-07T23:22:09.240862+0000","last_active":"2023-10-07T23:22:09.247996+0000","last_peered":"2023-10-07T23:22:09.247996+0000","last_clean":"2023-10-07T23:22:09.247996+0000","last_became_active":"2023-10-07T23:22:09.240353+0000","last_became_peered":"2023-10-07T23:22:09.240353+0000","last_unstale":"2023-10-07T23:22:09.247996+0000","last_undegraded":"2023-10-07T23:22:09.247996+0000","last_fullsized":"2023-10-07T23:22:09.247996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248424+0000","last_change":"2023-10-07T23:22:09.243291+0000","last_active":"2023-10-07T23:22:09.248424+0000","last_peered":"2023-10-07T23:22:09.248424+0000","last_clean":"2023-10-07T23:22:09.248424+0000","last_became_active":"2023-10-07T23:22:09.242972+0000","last_became_peered":"2023-10-07T23:22:09.242972+0000","last_unstale":"2023-10-07T23:22:09.248424+0000","last_undegraded":"2023-10-07T23:22:09.248424+0000","last_fullsized":"2023-10-07T23:22:09.248424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244674+0000","last_change":"2023-10-07T23:22:09.235853+0000","last_active":"2023-10-07T23:22:09.244674+0000","last_peered":"2023-10-07T23:22:09.244674+0000","last_clean":"2023-10-07T23:22:09.244674+0000","last_became_active":"2023-10-07T23:22:09.235686+0000","last_became_peered":"2023-10-07T23:22:09.235686+0000","last_unstale":"2023-10-07T23:22:09.244674+0000","last_undegraded":"2023-10-07T23:22:09.244674+0000","last_fullsized":"2023-10-07T23:22:09.244674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:51.866516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238240+0000","last_change":"2023-10-07T23:22:09.231694+0000","last_active":"2023-10-07T23:22:09.238240+0000","last_peered":"2023-10-07T23:22:09.238240+0000","last_clean":"2023-10-07T23:22:09.238240+0000","last_became_active":"2023-10-07T23:22:09.231199+0000","last_became_peered":"2023-10-07T23:22:09.231199+0000","last_unstale":"2023-10-07T23:22:09.238240+0000","last_undegraded":"2023-10-07T23:22:09.238240+0000","last_fullsized":"2023-10-07T23:22:09.238240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:04.106129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239600+0000","last_change":"2023-10-07T23:22:09.230322+0000","last_active":"2023-10-07T23:22:09.239600+0000","last_peered":"2023-10-07T23:22:09.239600+0000","last_clean":"2023-10-07T23:22:09.239600+0000","last_became_active":"2023-10-07T23:22:09.229441+0000","last_became_peered":"2023-10-07T23:22:09.229441+0000","last_unstale":"2023-10-07T23:22:09.239600+0000","last_undegraded":"2023-10-07T23:22:09.239600+0000","last_fullsized":"2023-10-07T23:22:09.239600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:22.309792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235790+0000","last_change":"2023-10-07T23:22:09.233463+0000","last_active":"2023-10-07T23:22:09.235790+0000","last_peered":"2023-10-07T23:22:09.235790+0000","last_clean":"2023-10-07T23:22:09.235790+0000","last_became_active":"2023-10-07T23:22:09.230666+0000","last_became_peered":"2023-10-07T23:22:09.230666+0000","last_unstale":"2023-10-07T23:22:09.235790+0000","last_undegraded":"2023-10-07T23:22:09.235790+0000","last_fullsized":"2023-10-07T23:22:09.235790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231066+0000","last_change":"2023-10-07T23:22:09.224898+0000","last_active":"2023-10-07T23:22:09.231066+0000","last_peered":"2023-10-07T23:22:09.231066+0000","last_clean":"2023-10-07T23:22:09.231066+0000","last_became_active":"2023-10-07T23:22:09.224592+0000","last_became_peered":"2023-10-07T23:22:09.224592+0000","last_unstale":"2023-10-07T23:22:09.231066+0000","last_undegraded":"2023-10-07T23:22:09.231066+0000","last_fullsized":"2023-10-07T23:22:09.231066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.242659+0000","last_change":"2023-10-07T23:22:09.237812+0000","last_active":"2023-10-07T23:22:09.242659+0000","last_peered":"2023-10-07T23:22:09.242659+0000","last_clean":"2023-10-07T23:22:09.242659+0000","last_became_active":"2023-10-07T23:22:09.237475+0000","last_became_peered":"2023-10-07T23:22:09.237475+0000","last_unstale":"2023-10-07T23:22:09.242659+0000","last_undegraded":"2023-10-07T23:22:09.242659+0000","last_fullsized":"2023-10-07T23:22:09.242659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:49.173670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244670+0000","last_change":"2023-10-07T23:22:09.236931+0000","last_active":"2023-10-07T23:22:09.244670+0000","last_peered":"2023-10-07T23:22:09.244670+0000","last_clean":"2023-10-07T23:22:09.244670+0000","last_became_active":"2023-10-07T23:22:09.236529+0000","last_became_peered":"2023-10-07T23:22:09.236529+0000","last_unstale":"2023-10-07T23:22:09.244670+0000","last_undegraded":"2023-10-07T23:22:09.244670+0000","last_fullsized":"2023-10-07T23:22:09.244670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:53:50.386927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248392+0000","last_change":"2023-10-07T23:22:09.243084+0000","last_active":"2023-10-07T23:22:09.248392+0000","last_peered":"2023-10-07T23:22:09.248392+0000","last_clean":"2023-10-07T23:22:09.248392+0000","last_became_active":"2023-10-07T23:22:09.242731+0000","last_became_peered":"2023-10-07T23:22:09.242731+0000","last_unstale":"2023-10-07T23:22:09.248392+0000","last_undegraded":"2023-10-07T23:22:09.248392+0000","last_fullsized":"2023-10-07T23:22:09.248392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:35.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.242813+0000","last_change":"2023-10-07T23:22:09.242028+0000","last_active":"2023-10-07T23:22:09.242813+0000","last_peered":"2023-10-07T23:22:09.242813+0000","last_clean":"2023-10-07T23:22:09.242813+0000","last_became_active":"2023-10-07T23:22:09.241684+0000","last_became_peered":"2023-10-07T23:22:09.241684+0000","last_unstale":"2023-10-07T23:22:09.242813+0000","last_undegraded":"2023-10-07T23:22:09.242813+0000","last_fullsized":"2023-10-07T23:22:09.242813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:29.967352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243496+0000","last_change":"2023-10-07T23:22:09.242444+0000","last_active":"2023-10-07T23:22:09.243496+0000","last_peered":"2023-10-07T23:22:09.243496+0000","last_clean":"2023-10-07T23:22:09.243496+0000","last_became_active":"2023-10-07T23:22:09.242008+0000","last_became_peered":"2023-10-07T23:22:09.242008+0000","last_unstale":"2023-10-07T23:22:09.243496+0000","last_undegraded":"2023-10-07T23:22:09.243496+0000","last_fullsized":"2023-10-07T23:22:09.243496+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:10.135861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244168+0000","last_change":"2023-10-07T23:22:09.234930+0000","last_active":"2023-10-07T23:22:09.244168+0000","last_peered":"2023-10-07T23:22:09.244168+0000","last_clean":"2023-10-07T23:22:09.244168+0000","last_became_active":"2023-10-07T23:22:09.229596+0000","last_became_peered":"2023-10-07T23:22:09.229596+0000","last_unstale":"2023-10-07T23:22:09.244168+0000","last_undegraded":"2023-10-07T23:22:09.244168+0000","last_fullsized":"2023-10-07T23:22:09.244168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:37.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.230258+0000","last_change":"2023-10-07T23:22:09.222600+0000","last_active":"2023-10-07T23:22:09.230258+0000","last_peered":"2023-10-07T23:22:09.230258+0000","last_clean":"2023-10-07T23:22:09.230258+0000","last_became_active":"2023-10-07T23:22:09.222314+0000","last_became_peered":"2023-10-07T23:22:09.222314+0000","last_unstale":"2023-10-07T23:22:09.230258+0000","last_undegraded":"2023-10-07T23:22:09.230258+0000","last_fullsized":"2023-10-07T23:22:09.230258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:35.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248418+0000","last_change":"2023-10-07T23:22:09.243142+0000","last_active":"2023-10-07T23:22:09.248418+0000","last_peered":"2023-10-07T23:22:09.248418+0000","last_clean":"2023-10-07T23:22:09.248418+0000","last_became_active":"2023-10-07T23:22:09.242789+0000","last_became_peered":"2023-10-07T23:22:09.242789+0000","last_unstale":"2023-10-07T23:22:09.248418+0000","last_undegraded":"2023-10-07T23:22:09.248418+0000","last_fullsized":"2023-10-07T23:22:09.248418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235636+0000","last_change":"2023-10-07T23:22:09.230550+0000","last_active":"2023-10-07T23:22:09.235636+0000","last_peered":"2023-10-07T23:22:09.235636+0000","last_clean":"2023-10-07T23:22:09.235636+0000","last_became_active":"2023-10-07T23:22:09.227637+0000","last_became_peered":"2023-10-07T23:22:09.227637+0000","last_unstale":"2023-10-07T23:22:09.235636+0000","last_undegraded":"2023-10-07T23:22:09.235636+0000","last_fullsized":"2023-10-07T23:22:09.235636+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:29.967352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247730+0000","last_change":"2023-10-07T23:22:09.240045+0000","last_active":"2023-10-07T23:22:09.247730+0000","last_peered":"2023-10-07T23:22:09.247730+0000","last_clean":"2023-10-07T23:22:09.247730+0000","last_became_active":"2023-10-07T23:22:09.236783+0000","last_became_peered":"2023-10-07T23:22:09.236783+0000","last_unstale":"2023-10-07T23:22:09.247730+0000","last_undegraded":"2023-10-07T23:22:09.247730+0000","last_fullsized":"2023-10-07T23:22:09.247730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:40.100405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248313+0000","last_change":"2023-10-07T23:22:09.242606+0000","last_active":"2023-10-07T23:22:09.248313+0000","last_peered":"2023-10-07T23:22:09.248313+0000","last_clean":"2023-10-07T23:22:09.248313+0000","last_became_active":"2023-10-07T23:22:09.241969+0000","last_became_peered":"2023-10-07T23:22:09.241969+0000","last_unstale":"2023-10-07T23:22:09.248313+0000","last_undegraded":"2023-10-07T23:22:09.248313+0000","last_fullsized":"2023-10-07T23:22:09.248313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:17.259019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.228809+0000","last_change":"2023-10-07T23:22:09.227795+0000","last_active":"2023-10-07T23:22:09.228809+0000","last_peered":"2023-10-07T23:22:09.228809+0000","last_clean":"2023-10-07T23:22:09.228809+0000","last_became_active":"2023-10-07T23:22:09.227335+0000","last_became_peered":"2023-10-07T23:22:09.227335+0000","last_unstale":"2023-10-07T23:22:09.228809+0000","last_undegraded":"2023-10-07T23:22:09.228809+0000","last_fullsized":"2023-10-07T23:22:09.228809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:10.401035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.213145+0000","last_change":"2023-10-07T23:22:09.211146+0000","last_active":"2023-10-07T23:22:09.213145+0000","last_peered":"2023-10-07T23:22:09.213145+0000","last_clean":"2023-10-07T23:22:09.213145+0000","last_became_active":"2023-10-07T23:22:09.210702+0000","last_became_peered":"2023-10-07T23:22:09.210702+0000","last_unstale":"2023-10-07T23:22:09.213145+0000","last_undegraded":"2023-10-07T23:22:09.213145+0000","last_fullsized":"2023-10-07T23:22:09.213145+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243046+0000","last_change":"2023-10-07T23:22:09.238893+0000","last_active":"2023-10-07T23:22:09.243046+0000","last_peered":"2023-10-07T23:22:09.243046+0000","last_clean":"2023-10-07T23:22:09.243046+0000","last_became_active":"2023-10-07T23:22:09.238549+0000","last_became_peered":"2023-10-07T23:22:09.238549+0000","last_unstale":"2023-10-07T23:22:09.243046+0000","last_undegraded":"2023-10-07T23:22:09.243046+0000","last_fullsized":"2023-10-07T23:22:09.243046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:43:18.264986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.229062+0000","last_change":"2023-10-07T23:22:09.224304+0000","last_active":"2023-10-07T23:22:09.229062+0000","last_peered":"2023-10-07T23:22:09.229062+0000","last_clean":"2023-10-07T23:22:09.229062+0000","last_became_active":"2023-10-07T23:22:09.223992+0000","last_became_peered":"2023-10-07T23:22:09.223992+0000","last_unstale":"2023-10-07T23:22:09.229062+0000","last_undegraded":"2023-10-07T23:22:09.229062+0000","last_fullsized":"2023-10-07T23:22:09.229062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:44.258733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232065+0000","last_change":"2023-10-07T23:22:09.225439+0000","last_active":"2023-10-07T23:22:09.232065+0000","last_peered":"2023-10-07T23:22:09.232065+0000","last_clean":"2023-10-07T23:22:09.232065+0000","last_became_active":"2023-10-07T23:22:09.225050+0000","last_became_peered":"2023-10-07T23:22:09.225050+0000","last_unstale":"2023-10-07T23:22:09.232065+0000","last_undegraded":"2023-10-07T23:22:09.232065+0000","last_fullsized":"2023-10-07T23:22:09.232065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:59.536276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244494+0000","last_change":"2023-10-07T23:22:09.236386+0000","last_active":"2023-10-07T23:22:09.244494+0000","last_peered":"2023-10-07T23:22:09.244494+0000","last_clean":"2023-10-07T23:22:09.244494+0000","last_became_active":"2023-10-07T23:22:09.236100+0000","last_became_peered":"2023-10-07T23:22:09.236100+0000","last_unstale":"2023-10-07T23:22:09.244494+0000","last_undegraded":"2023-10-07T23:22:09.244494+0000","last_fullsized":"2023-10-07T23:22:09.244494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:27.578614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245103+0000","last_change":"2023-10-07T23:22:09.230696+0000","last_active":"2023-10-07T23:22:09.245103+0000","last_peered":"2023-10-07T23:22:09.245103+0000","last_clean":"2023-10-07T23:22:09.245103+0000","last_became_active":"2023-10-07T23:22:09.230360+0000","last_became_peered":"2023-10-07T23:22:09.230360+0000","last_unstale":"2023-10-07T23:22:09.245103+0000","last_undegraded":"2023-10-07T23:22:09.245103+0000","last_fullsized":"2023-10-07T23:22:09.245103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:13.198994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245064+0000","last_change":"2023-10-07T23:22:09.237673+0000","last_active":"2023-10-07T23:22:09.245064+0000","last_peered":"2023-10-07T23:22:09.245064+0000","last_clean":"2023-10-07T23:22:09.245064+0000","last_became_active":"2023-10-07T23:22:09.236679+0000","last_became_peered":"2023-10-07T23:22:09.236679+0000","last_unstale":"2023-10-07T23:22:09.245064+0000","last_undegraded":"2023-10-07T23:22:09.245064+0000","last_fullsized":"2023-10-07T23:22:09.245064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:10.075644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232268+0000","last_change":"2023-10-07T23:22:09.225102+0000","last_active":"2023-10-07T23:22:09.232268+0000","last_peered":"2023-10-07T23:22:09.232268+0000","last_clean":"2023-10-07T23:22:09.232268+0000","last_became_active":"2023-10-07T23:22:09.224760+0000","last_became_peered":"2023-10-07T23:22:09.224760+0000","last_unstale":"2023-10-07T23:22:09.232268+0000","last_undegraded":"2023-10-07T23:22:09.232268+0000","last_fullsized":"2023-10-07T23:22:09.232268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:40.100405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237386+0000","last_change":"2023-10-07T23:22:09.232606+0000","last_active":"2023-10-07T23:22:09.237386+0000","last_peered":"2023-10-07T23:22:09.237386+0000","last_clean":"2023-10-07T23:22:09.237386+0000","last_became_active":"2023-10-07T23:22:09.232274+0000","last_became_peered":"2023-10-07T23:22:09.232274+0000","last_unstale":"2023-10-07T23:22:09.237386+0000","last_undegraded":"2023-10-07T23:22:09.237386+0000","last_fullsized":"2023-10-07T23:22:09.237386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:50.522855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.234693+0000","last_change":"2023-10-07T23:22:09.229905+0000","last_active":"2023-10-07T23:22:09.234693+0000","last_peered":"2023-10-07T23:22:09.234693+0000","last_clean":"2023-10-07T23:22:09.234693+0000","last_became_active":"2023-10-07T23:22:09.227574+0000","last_became_peered":"2023-10-07T23:22:09.227574+0000","last_unstale":"2023-10-07T23:22:09.234693+0000","last_undegraded":"2023-10-07T23:22:09.234693+0000","last_fullsized":"2023-10-07T23:22:09.234693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:40:02.590917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.226692+0000","last_change":"2023-10-07T23:22:09.223398+0000","last_active":"2023-10-07T23:22:09.226692+0000","last_peered":"2023-10-07T23:22:09.226692+0000","last_clean":"2023-10-07T23:22:09.226692+0000","last_became_active":"2023-10-07T23:22:09.222975+0000","last_became_peered":"2023-10-07T23:22:09.222975+0000","last_unstale":"2023-10-07T23:22:09.226692+0000","last_undegraded":"2023-10-07T23:22:09.226692+0000","last_fullsized":"2023-10-07T23:22:09.226692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231085+0000","last_change":"2023-10-07T23:22:09.225723+0000","last_active":"2023-10-07T23:22:09.231085+0000","last_peered":"2023-10-07T23:22:09.231085+0000","last_clean":"2023-10-07T23:22:09.231085+0000","last_became_active":"2023-10-07T23:22:09.225153+0000","last_became_peered":"2023-10-07T23:22:09.225153+0000","last_unstale":"2023-10-07T23:22:09.231085+0000","last_undegraded":"2023-10-07T23:22:09.231085+0000","last_fullsized":"2023-10-07T23:22:09.231085+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:14.039148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.228763+0000","last_change":"2023-10-07T23:22:09.224278+0000","last_active":"2023-10-07T23:22:09.228763+0000","last_peered":"2023-10-07T23:22:09.228763+0000","last_clean":"2023-10-07T23:22:09.228763+0000","last_became_active":"2023-10-07T23:22:09.223924+0000","last_became_peered":"2023-10-07T23:22:09.223924+0000","last_unstale":"2023-10-07T23:22:09.228763+0000","last_undegraded":"2023-10-07T23:22:09.228763+0000","last_fullsized":"2023-10-07T23:22:09.228763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.234482+0000","last_change":"2023-10-07T23:22:08.196666+0000","last_active":"2023-10-07T23:22:09.234482+0000","last_peered":"2023-10-07T23:22:09.234482+0000","last_clean":"2023-10-07T23:22:09.234482+0000","last_became_active":"2023-10-07T23:22:08.196325+0000","last_became_peered":"2023-10-07T23:22:08.196325+0000","last_unstale":"2023-10-07T23:22:09.234482+0000","last_undegraded":"2023-10-07T23:22:09.234482+0000","last_fullsized":"2023-10-07T23:22:09.234482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:24.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.211766+0000","last_change":"2023-10-07T23:22:09.210142+0000","last_active":"2023-10-07T23:22:09.211766+0000","last_peered":"2023-10-07T23:22:09.211766+0000","last_clean":"2023-10-07T23:22:09.211766+0000","last_became_active":"2023-10-07T23:22:09.209843+0000","last_became_peered":"2023-10-07T23:22:09.209843+0000","last_unstale":"2023-10-07T23:22:09.211766+0000","last_undegraded":"2023-10-07T23:22:09.211766+0000","last_fullsized":"2023-10-07T23:22:09.211766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:04.648785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236175+0000","last_change":"2023-10-07T23:22:08.199116+0000","last_active":"2023-10-07T23:22:09.236175+0000","last_peered":"2023-10-07T23:22:09.236175+0000","last_clean":"2023-10-07T23:22:09.236175+0000","last_became_active":"2023-10-07T23:22:08.198514+0000","last_became_peered":"2023-10-07T23:22:08.198514+0000","last_unstale":"2023-10-07T23:22:09.236175+0000","last_undegraded":"2023-10-07T23:22:09.236175+0000","last_fullsized":"2023-10-07T23:22:09.236175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:09.387655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245572+0000","last_change":"2023-10-07T23:22:09.241884+0000","last_active":"2023-10-07T23:22:09.245572+0000","last_peered":"2023-10-07T23:22:09.245572+0000","last_clean":"2023-10-07T23:22:09.245572+0000","last_became_active":"2023-10-07T23:22:09.241629+0000","last_became_peered":"2023-10-07T23:22:09.241629+0000","last_unstale":"2023-10-07T23:22:09.245572+0000","last_undegraded":"2023-10-07T23:22:09.245572+0000","last_fullsized":"2023-10-07T23:22:09.245572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:44.258733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245333+0000","last_change":"2023-10-07T23:22:08.197599+0000","last_active":"2023-10-07T23:22:09.245333+0000","last_peered":"2023-10-07T23:22:09.245333+0000","last_clean":"2023-10-07T23:22:09.245333+0000","last_became_active":"2023-10-07T23:22:08.197276+0000","last_became_peered":"2023-10-07T23:22:08.197276+0000","last_unstale":"2023-10-07T23:22:09.245333+0000","last_undegraded":"2023-10-07T23:22:09.245333+0000","last_fullsized":"2023-10-07T23:22:09.245333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:43.245354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244594+0000","last_change":"2023-10-07T23:22:09.236900+0000","last_active":"2023-10-07T23:22:09.244594+0000","last_peered":"2023-10-07T23:22:09.244594+0000","last_clean":"2023-10-07T23:22:09.244594+0000","last_became_active":"2023-10-07T23:22:09.236202+0000","last_became_peered":"2023-10-07T23:22:09.236202+0000","last_unstale":"2023-10-07T23:22:09.244594+0000","last_undegraded":"2023-10-07T23:22:09.244594+0000","last_fullsized":"2023-10-07T23:22:09.244594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:34:11.022020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244641+0000","last_change":"2023-10-07T23:22:08.200777+0000","last_active":"2023-10-07T23:22:09.244641+0000","last_peered":"2023-10-07T23:22:09.244641+0000","last_clean":"2023-10-07T23:22:09.244641+0000","last_became_active":"2023-10-07T23:22:08.200508+0000","last_became_peered":"2023-10-07T23:22:08.200508+0000","last_unstale":"2023-10-07T23:22:09.244641+0000","last_undegraded":"2023-10-07T23:22:09.244641+0000","last_fullsized":"2023-10-07T23:22:09.244641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:39.410643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235598+0000","last_change":"2023-10-07T23:22:09.230430+0000","last_active":"2023-10-07T23:22:09.235598+0000","last_peered":"2023-10-07T23:22:09.235598+0000","last_clean":"2023-10-07T23:22:09.235598+0000","last_became_active":"2023-10-07T23:22:09.230173+0000","last_became_peered":"2023-10-07T23:22:09.230173+0000","last_unstale":"2023-10-07T23:22:09.235598+0000","last_undegraded":"2023-10-07T23:22:09.235598+0000","last_fullsized":"2023-10-07T23:22:09.235598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:22.025077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.246332+0000","last_change":"2023-10-07T23:22:08.195534+0000","last_active":"2023-10-07T23:22:09.246332+0000","last_peered":"2023-10-07T23:22:09.246332+0000","last_clean":"2023-10-07T23:22:09.246332+0000","last_became_active":"2023-10-07T23:22:08.193314+0000","last_became_peered":"2023-10-07T23:22:08.193314+0000","last_unstale":"2023-10-07T23:22:09.246332+0000","last_undegraded":"2023-10-07T23:22:09.246332+0000","last_fullsized":"2023-10-07T23:22:09.246332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:24.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232228+0000","last_change":"2023-10-07T23:22:09.226598+0000","last_active":"2023-10-07T23:22:09.232228+0000","last_peered":"2023-10-07T23:22:09.232228+0000","last_clean":"2023-10-07T23:22:09.232228+0000","last_became_active":"2023-10-07T23:22:09.226207+0000","last_became_peered":"2023-10-07T23:22:09.226207+0000","last_unstale":"2023-10-07T23:22:09.232228+0000","last_undegraded":"2023-10-07T23:22:09.232228+0000","last_fullsized":"2023-10-07T23:22:09.232228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245464+0000","last_change":"2023-10-07T23:22:08.193388+0000","last_active":"2023-10-07T23:22:09.245464+0000","last_peered":"2023-10-07T23:22:09.245464+0000","last_clean":"2023-10-07T23:22:09.245464+0000","last_became_active":"2023-10-07T23:22:08.193083+0000","last_became_peered":"2023-10-07T23:22:08.193083+0000","last_unstale":"2023-10-07T23:22:09.245464+0000","last_undegraded":"2023-10-07T23:22:09.245464+0000","last_fullsized":"2023-10-07T23:22:09.245464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:41.116405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.212184+0000","last_change":"2023-10-07T23:22:09.210142+0000","last_active":"2023-10-07T23:22:09.212184+0000","last_peered":"2023-10-07T23:22:09.212184+0000","last_clean":"2023-10-07T23:22:09.212184+0000","last_became_active":"2023-10-07T23:22:09.209698+0000","last_became_peered":"2023-10-07T23:22:09.209698+0000","last_unstale":"2023-10-07T23:22:09.212184+0000","last_undegraded":"2023-10-07T23:22:09.212184+0000","last_fullsized":"2023-10-07T23:22:09.212184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217700+0000","last_change":"2023-10-07T23:22:08.196651+0000","last_active":"2023-10-07T23:22:09.217700+0000","last_peered":"2023-10-07T23:22:09.217700+0000","last_clean":"2023-10-07T23:22:09.217700+0000","last_became_active":"2023-10-07T23:22:08.196246+0000","last_became_peered":"2023-10-07T23:22:08.196246+0000","last_unstale":"2023-10-07T23:22:09.217700+0000","last_undegraded":"2023-10-07T23:22:09.217700+0000","last_fullsized":"2023-10-07T23:22:09.217700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:05.186215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.241112+0000","last_change":"2023-10-07T23:22:09.229068+0000","last_active":"2023-10-07T23:22:09.241112+0000","last_peered":"2023-10-07T23:22:09.241112+0000","last_clean":"2023-10-07T23:22:09.241112+0000","last_became_active":"2023-10-07T23:22:09.228732+0000","last_became_peered":"2023-10-07T23:22:09.228732+0000","last_unstale":"2023-10-07T23:22:09.241112+0000","last_undegraded":"2023-10-07T23:22:09.241112+0000","last_fullsized":"2023-10-07T23:22:09.241112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:49:53.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.205608+0000","last_change":"2023-10-07T23:22:08.189598+0000","last_active":"2023-10-07T23:22:09.205608+0000","last_peered":"2023-10-07T23:22:09.205608+0000","last_clean":"2023-10-07T23:22:09.205608+0000","last_became_active":"2023-10-07T23:22:08.189266+0000","last_became_peered":"2023-10-07T23:22:08.189266+0000","last_unstale":"2023-10-07T23:22:09.205608+0000","last_undegraded":"2023-10-07T23:22:09.205608+0000","last_fullsized":"2023-10-07T23:22:09.205608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:34:10.974537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238833+0000","last_change":"2023-10-07T23:22:09.237925+0000","last_active":"2023-10-07T23:22:09.238833+0000","last_peered":"2023-10-07T23:22:09.238833+0000","last_clean":"2023-10-07T23:22:09.238833+0000","last_became_active":"2023-10-07T23:22:09.237618+0000","last_became_peered":"2023-10-07T23:22:09.237618+0000","last_unstale":"2023-10-07T23:22:09.238833+0000","last_undegraded":"2023-10-07T23:22:09.238833+0000","last_fullsized":"2023-10-07T23:22:09.238833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.218106+0000","last_change":"2023-10-07T23:22:08.197437+0000","last_active":"2023-10-07T23:22:09.218106+0000","last_peered":"2023-10-07T23:22:09.218106+0000","last_clean":"2023-10-07T23:22:09.218106+0000","last_became_active":"2023-10-07T23:22:08.197130+0000","last_became_peered":"2023-10-07T23:22:08.197130+0000","last_unstale":"2023-10-07T23:22:09.218106+0000","last_undegraded":"2023-10-07T23:22:09.218106+0000","last_fullsized":"2023-10-07T23:22:09.218106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:03.635405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248272+0000","last_change":"2023-10-07T23:22:09.242371+0000","last_active":"2023-10-07T23:22:09.248272+0000","last_peered":"2023-10-07T23:22:09.248272+0000","last_clean":"2023-10-07T23:22:09.248272+0000","last_became_active":"2023-10-07T23:22:09.241987+0000","last_became_peered":"2023-10-07T23:22:09.241987+0000","last_unstale":"2023-10-07T23:22:09.248272+0000","last_undegraded":"2023-10-07T23:22:09.248272+0000","last_fullsized":"2023-10-07T23:22:09.248272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.204570+0000","last_change":"2023-10-07T23:22:08.192642+0000","last_active":"2023-10-07T23:22:09.204570+0000","last_peered":"2023-10-07T23:22:09.204570+0000","last_clean":"2023-10-07T23:22:09.204570+0000","last_became_active":"2023-10-07T23:22:08.192294+0000","last_became_peered":"2023-10-07T23:22:08.192294+0000","last_unstale":"2023-10-07T23:22:09.204570+0000","last_undegraded":"2023-10-07T23:22:09.204570+0000","last_fullsized":"2023-10-07T23:22:09.204570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:09.122481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247772+0000","last_change":"2023-10-07T23:22:09.240062+0000","last_active":"2023-10-07T23:22:09.247772+0000","last_peered":"2023-10-07T23:22:09.247772+0000","last_clean":"2023-10-07T23:22:09.247772+0000","last_became_active":"2023-10-07T23:22:09.239690+0000","last_became_peered":"2023-10-07T23:22:09.239690+0000","last_unstale":"2023-10-07T23:22:09.247772+0000","last_undegraded":"2023-10-07T23:22:09.247772+0000","last_fullsized":"2023-10-07T23:22:09.247772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:20.669822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237132+0000","last_change":"2023-10-07T23:22:08.199948+0000","last_active":"2023-10-07T23:22:09.237132+0000","last_peered":"2023-10-07T23:22:09.237132+0000","last_clean":"2023-10-07T23:22:09.237132+0000","last_became_active":"2023-10-07T23:22:08.199679+0000","last_became_peered":"2023-10-07T23:22:08.199679+0000","last_unstale":"2023-10-07T23:22:09.237132+0000","last_undegraded":"2023-10-07T23:22:09.237132+0000","last_fullsized":"2023-10-07T23:22:09.237132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:40.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.251381+0000","last_change":"2023-10-07T23:22:09.246678+0000","last_active":"2023-10-07T23:22:09.251381+0000","last_peered":"2023-10-07T23:22:09.251381+0000","last_clean":"2023-10-07T23:22:09.251381+0000","last_became_active":"2023-10-07T23:22:09.246396+0000","last_became_peered":"2023-10-07T23:22:09.246396+0000","last_unstale":"2023-10-07T23:22:09.251381+0000","last_undegraded":"2023-10-07T23:22:09.251381+0000","last_fullsized":"2023-10-07T23:22:09.251381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.250603+0000","last_change":"2023-10-07T23:22:08.197478+0000","last_active":"2023-10-07T23:22:09.250603+0000","last_peered":"2023-10-07T23:22:09.250603+0000","last_clean":"2023-10-07T23:22:09.250603+0000","last_became_active":"2023-10-07T23:22:08.197183+0000","last_became_peered":"2023-10-07T23:22:08.197183+0000","last_unstale":"2023-10-07T23:22:09.250603+0000","last_undegraded":"2023-10-07T23:22:09.250603+0000","last_fullsized":"2023-10-07T23:22:09.250603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:13.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.241089+0000","last_change":"2023-10-07T23:22:09.229006+0000","last_active":"2023-10-07T23:22:09.241089+0000","last_peered":"2023-10-07T23:22:09.241089+0000","last_clean":"2023-10-07T23:22:09.241089+0000","last_became_active":"2023-10-07T23:22:09.228711+0000","last_became_peered":"2023-10-07T23:22:09.228711+0000","last_unstale":"2023-10-07T23:22:09.241089+0000","last_undegraded":"2023-10-07T23:22:09.241089+0000","last_fullsized":"2023-10-07T23:22:09.241089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:53.104146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238362+0000","last_change":"2023-10-07T23:22:08.190945+0000","last_active":"2023-10-07T23:22:09.238362+0000","last_peered":"2023-10-07T23:22:09.238362+0000","last_clean":"2023-10-07T23:22:09.238362+0000","last_became_active":"2023-10-07T23:22:08.190614+0000","last_became_peered":"2023-10-07T23:22:08.190614+0000","last_unstale":"2023-10-07T23:22:09.238362+0000","last_undegraded":"2023-10-07T23:22:09.238362+0000","last_fullsized":"2023-10-07T23:22:09.238362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:20.764751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245047+0000","last_change":"2023-10-07T23:22:09.236038+0000","last_active":"2023-10-07T23:22:09.245047+0000","last_peered":"2023-10-07T23:22:09.245047+0000","last_clean":"2023-10-07T23:22:09.245047+0000","last_became_active":"2023-10-07T23:22:09.235746+0000","last_became_peered":"2023-10-07T23:22:09.235746+0000","last_unstale":"2023-10-07T23:22:09.245047+0000","last_undegraded":"2023-10-07T23:22:09.245047+0000","last_fullsized":"2023-10-07T23:22:09.245047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.214906+0000","last_change":"2023-10-07T23:22:08.197284+0000","last_active":"2023-10-07T23:22:09.214906+0000","last_peered":"2023-10-07T23:22:09.214906+0000","last_clean":"2023-10-07T23:22:09.214906+0000","last_became_active":"2023-10-07T23:22:08.196962+0000","last_became_peered":"2023-10-07T23:22:08.196962+0000","last_unstale":"2023-10-07T23:22:09.214906+0000","last_undegraded":"2023-10-07T23:22:09.214906+0000","last_fullsized":"2023-10-07T23:22:09.214906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:04:34.498041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231103+0000","last_change":"2023-10-07T23:22:09.225903+0000","last_active":"2023-10-07T23:22:09.231103+0000","last_peered":"2023-10-07T23:22:09.231103+0000","last_clean":"2023-10-07T23:22:09.231103+0000","last_became_active":"2023-10-07T23:22:09.225462+0000","last_became_peered":"2023-10-07T23:22:09.225462+0000","last_unstale":"2023-10-07T23:22:09.231103+0000","last_undegraded":"2023-10-07T23:22:09.231103+0000","last_fullsized":"2023-10-07T23:22:09.231103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:59.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217595+0000","last_change":"2023-10-07T23:22:08.199464+0000","last_active":"2023-10-07T23:22:09.217595+0000","last_peered":"2023-10-07T23:22:09.217595+0000","last_clean":"2023-10-07T23:22:09.217595+0000","last_became_active":"2023-10-07T23:22:08.199076+0000","last_became_peered":"2023-10-07T23:22:08.199076+0000","last_unstale":"2023-10-07T23:22:09.217595+0000","last_undegraded":"2023-10-07T23:22:09.217595+0000","last_fullsized":"2023-10-07T23:22:09.217595+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:43:17.251607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.211795+0000","last_change":"2023-10-07T23:22:09.210427+0000","last_active":"2023-10-07T23:22:09.211795+0000","last_peered":"2023-10-07T23:22:09.211795+0000","last_clean":"2023-10-07T23:22:09.211795+0000","last_became_active":"2023-10-07T23:22:09.210044+0000","last_became_peered":"2023-10-07T23:22:09.210044+0000","last_unstale":"2023-10-07T23:22:09.211795+0000","last_undegraded":"2023-10-07T23:22:09.211795+0000","last_fullsized":"2023-10-07T23:22:09.211795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248449+0000","last_change":"2023-10-07T23:22:08.201034+0000","last_active":"2023-10-07T23:22:09.248449+0000","last_peered":"2023-10-07T23:22:09.248449+0000","last_clean":"2023-10-07T23:22:09.248449+0000","last_became_active":"2023-10-07T23:22:08.200733+0000","last_became_peered":"2023-10-07T23:22:08.200733+0000","last_unstale":"2023-10-07T23:22:09.248449+0000","last_undegraded":"2023-10-07T23:22:09.248449+0000","last_fullsized":"2023-10-07T23:22:09.248449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:19.750302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236063+0000","last_change":"2023-10-07T23:22:09.231189+0000","last_active":"2023-10-07T23:22:09.236063+0000","last_peered":"2023-10-07T23:22:09.236063+0000","last_clean":"2023-10-07T23:22:09.236063+0000","last_became_active":"2023-10-07T23:22:09.230905+0000","last_became_peered":"2023-10-07T23:22:09.230905+0000","last_unstale":"2023-10-07T23:22:09.236063+0000","last_undegraded":"2023-10-07T23:22:09.236063+0000","last_fullsized":"2023-10-07T23:22:09.236063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247875+0000","last_change":"2023-10-07T23:22:08.199006+0000","last_active":"2023-10-07T23:22:09.247875+0000","last_peered":"2023-10-07T23:22:09.247875+0000","last_clean":"2023-10-07T23:22:09.247875+0000","last_became_active":"2023-10-07T23:22:08.198687+0000","last_became_peered":"2023-10-07T23:22:08.198687+0000","last_unstale":"2023-10-07T23:22:09.247875+0000","last_undegraded":"2023-10-07T23:22:09.247875+0000","last_fullsized":"2023-10-07T23:22:09.247875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:05.995895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232256+0000","last_change":"2023-10-07T23:22:09.227299+0000","last_active":"2023-10-07T23:22:09.232256+0000","last_peered":"2023-10-07T23:22:09.232256+0000","last_clean":"2023-10-07T23:22:09.232256+0000","last_became_active":"2023-10-07T23:22:09.227018+0000","last_became_peered":"2023-10-07T23:22:09.227018+0000","last_unstale":"2023-10-07T23:22:09.232256+0000","last_undegraded":"2023-10-07T23:22:09.232256+0000","last_fullsized":"2023-10-07T23:22:09.232256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:43.237723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238398+0000","last_change":"2023-10-07T23:22:08.190155+0000","last_active":"2023-10-07T23:22:09.238398+0000","last_peered":"2023-10-07T23:22:09.238398+0000","last_clean":"2023-10-07T23:22:09.238398+0000","last_became_active":"2023-10-07T23:22:08.189890+0000","last_became_peered":"2023-10-07T23:22:08.189890+0000","last_unstale":"2023-10-07T23:22:09.238398+0000","last_undegraded":"2023-10-07T23:22:09.238398+0000","last_fullsized":"2023-10-07T23:22:09.238398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:37.585931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243922+0000","last_change":"2023-10-07T23:22:09.233095+0000","last_active":"2023-10-07T23:22:09.243922+0000","last_peered":"2023-10-07T23:22:09.243922+0000","last_clean":"2023-10-07T23:22:09.243922+0000","last_became_active":"2023-10-07T23:22:09.232792+0000","last_became_peered":"2023-10-07T23:22:09.232792+0000","last_unstale":"2023-10-07T23:22:09.243922+0000","last_undegraded":"2023-10-07T23:22:09.243922+0000","last_fullsized":"2023-10-07T23:22:09.243922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:06.539033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239500+0000","last_change":"2023-10-07T23:22:08.196526+0000","last_active":"2023-10-07T23:22:09.239500+0000","last_peered":"2023-10-07T23:22:09.239500+0000","last_clean":"2023-10-07T23:22:09.239500+0000","last_became_active":"2023-10-07T23:22:08.196260+0000","last_became_peered":"2023-10-07T23:22:08.196260+0000","last_unstale":"2023-10-07T23:22:09.239500+0000","last_undegraded":"2023-10-07T23:22:09.239500+0000","last_fullsized":"2023-10-07T23:22:09.239500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232300+0000","last_change":"2023-10-07T23:22:09.230763+0000","last_active":"2023-10-07T23:22:09.232300+0000","last_peered":"2023-10-07T23:22:09.232300+0000","last_clean":"2023-10-07T23:22:09.232300+0000","last_became_active":"2023-10-07T23:22:09.230487+0000","last_became_peered":"2023-10-07T23:22:09.230487+0000","last_unstale":"2023-10-07T23:22:09.232300+0000","last_undegraded":"2023-10-07T23:22:09.232300+0000","last_fullsized":"2023-10-07T23:22:09.232300+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:48.735570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.204980+0000","last_change":"2023-10-07T23:22:08.195897+0000","last_active":"2023-10-07T23:22:09.204980+0000","last_peered":"2023-10-07T23:22:09.204980+0000","last_clean":"2023-10-07T23:22:09.204980+0000","last_became_active":"2023-10-07T23:22:08.193256+0000","last_became_peered":"2023-10-07T23:22:08.193256+0000","last_unstale":"2023-10-07T23:22:09.204980+0000","last_undegraded":"2023-10-07T23:22:09.204980+0000","last_fullsized":"2023-10-07T23:22:09.204980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:39.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.225132+0000","last_change":"2023-10-07T23:22:09.222015+0000","last_active":"2023-10-07T23:22:09.225132+0000","last_peered":"2023-10-07T23:22:09.225132+0000","last_clean":"2023-10-07T23:22:09.225132+0000","last_became_active":"2023-10-07T23:22:09.221557+0000","last_became_peered":"2023-10-07T23:22:09.221557+0000","last_unstale":"2023-10-07T23:22:09.225132+0000","last_undegraded":"2023-10-07T23:22:09.225132+0000","last_fullsized":"2023-10-07T23:22:09.225132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:15.766802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243210+0000","last_change":"2023-10-07T23:22:08.200498+0000","last_active":"2023-10-07T23:22:09.243210+0000","last_peered":"2023-10-07T23:22:09.243210+0000","last_clean":"2023-10-07T23:22:09.243210+0000","last_became_active":"2023-10-07T23:22:08.200099+0000","last_became_peered":"2023-10-07T23:22:08.200099+0000","last_unstale":"2023-10-07T23:22:09.243210+0000","last_undegraded":"2023-10-07T23:22:09.243210+0000","last_fullsized":"2023-10-07T23:22:09.243210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:03.092749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238259+0000","last_change":"2023-10-07T23:22:09.232479+0000","last_active":"2023-10-07T23:22:09.238259+0000","last_peered":"2023-10-07T23:22:09.238259+0000","last_clean":"2023-10-07T23:22:09.238259+0000","last_became_active":"2023-10-07T23:22:09.232186+0000","last_became_peered":"2023-10-07T23:22:09.232186+0000","last_unstale":"2023-10-07T23:22:09.238259+0000","last_undegraded":"2023-10-07T23:22:09.238259+0000","last_fullsized":"2023-10-07T23:22:09.238259+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:56.442036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.208549+0000","last_change":"2023-10-07T23:22:08.200014+0000","last_active":"2023-10-07T23:22:09.208549+0000","last_peered":"2023-10-07T23:22:09.208549+0000","last_clean":"2023-10-07T23:22:09.208549+0000","last_became_active":"2023-10-07T23:22:08.199729+0000","last_became_peered":"2023-10-07T23:22:08.199729+0000","last_unstale":"2023-10-07T23:22:09.208549+0000","last_undegraded":"2023-10-07T23:22:09.208549+0000","last_fullsized":"2023-10-07T23:22:09.208549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.223392+0000","last_change":"2023-10-07T23:22:09.221769+0000","last_active":"2023-10-07T23:22:09.223392+0000","last_peered":"2023-10-07T23:22:09.223392+0000","last_clean":"2023-10-07T23:22:09.223392+0000","last_became_active":"2023-10-07T23:22:09.221362+0000","last_became_peered":"2023-10-07T23:22:09.221362+0000","last_unstale":"2023-10-07T23:22:09.223392+0000","last_undegraded":"2023-10-07T23:22:09.223392+0000","last_fullsized":"2023-10-07T23:22:09.223392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:40.217782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.205982+0000","last_change":"2023-10-07T23:22:08.202898+0000","last_active":"2023-10-07T23:22:09.205982+0000","last_peered":"2023-10-07T23:22:09.205982+0000","last_clean":"2023-10-07T23:22:09.205982+0000","last_became_active":"2023-10-07T23:22:08.202581+0000","last_became_peered":"2023-10-07T23:22:08.202581+0000","last_unstale":"2023-10-07T23:22:09.205982+0000","last_undegraded":"2023-10-07T23:22:09.205982+0000","last_fullsized":"2023-10-07T23:22:09.205982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:14.753423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235805+0000","last_change":"2023-10-07T23:22:09.233869+0000","last_active":"2023-10-07T23:22:09.235805+0000","last_peered":"2023-10-07T23:22:09.235805+0000","last_clean":"2023-10-07T23:22:09.235805+0000","last_became_active":"2023-10-07T23:22:09.230652+0000","last_became_peered":"2023-10-07T23:22:09.230652+0000","last_unstale":"2023-10-07T23:22:09.235805+0000","last_undegraded":"2023-10-07T23:22:09.235805+0000","last_fullsized":"2023-10-07T23:22:09.235805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:43.237723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.218680+0000","last_change":"2023-10-07T23:22:08.199889+0000","last_active":"2023-10-07T23:22:09.218680+0000","last_peered":"2023-10-07T23:22:09.218680+0000","last_clean":"2023-10-07T23:22:09.218680+0000","last_became_active":"2023-10-07T23:22:08.199599+0000","last_became_peered":"2023-10-07T23:22:08.199599+0000","last_unstale":"2023-10-07T23:22:09.218680+0000","last_undegraded":"2023-10-07T23:22:09.218680+0000","last_fullsized":"2023-10-07T23:22:09.218680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:03.635405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238106+0000","last_change":"2023-10-07T23:22:09.230999+0000","last_active":"2023-10-07T23:22:09.238106+0000","last_peered":"2023-10-07T23:22:09.238106+0000","last_clean":"2023-10-07T23:22:09.238106+0000","last_became_active":"2023-10-07T23:22:09.229998+0000","last_became_peered":"2023-10-07T23:22:09.229998+0000","last_unstale":"2023-10-07T23:22:09.238106+0000","last_undegraded":"2023-10-07T23:22:09.238106+0000","last_fullsized":"2023-10-07T23:22:09.238106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:25.650167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243922+0000","last_change":"2023-10-07T23:22:08.198108+0000","last_active":"2023-10-07T23:22:09.243922+0000","last_peered":"2023-10-07T23:22:09.243922+0000","last_clean":"2023-10-07T23:22:09.243922+0000","last_became_active":"2023-10-07T23:22:08.197784+0000","last_became_peered":"2023-10-07T23:22:08.197784+0000","last_unstale":"2023-10-07T23:22:09.243922+0000","last_undegraded":"2023-10-07T23:22:09.243922+0000","last_fullsized":"2023-10-07T23:22:09.243922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:27.339348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248455+0000","last_change":"2023-10-07T23:22:09.243462+0000","last_active":"2023-10-07T23:22:09.248455+0000","last_peered":"2023-10-07T23:22:09.248455+0000","last_clean":"2023-10-07T23:22:09.248455+0000","last_became_active":"2023-10-07T23:22:09.243027+0000","last_became_peered":"2023-10-07T23:22:09.243027+0000","last_unstale":"2023-10-07T23:22:09.248455+0000","last_undegraded":"2023-10-07T23:22:09.248455+0000","last_fullsized":"2023-10-07T23:22:09.248455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:40:02.590917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-07T23:22:10.330210+0000","last_change":"2023-10-07T23:22:08.200713+0000","last_active":"2023-10-07T23:22:10.330210+0000","last_peered":"2023-10-07T23:22:10.330210+0000","last_clean":"2023-10-07T23:22:10.330210+0000","last_became_active":"2023-10-07T23:22:08.200417+0000","last_became_peered":"2023-10-07T23:22:08.200417+0000","last_unstale":"2023-10-07T23:22:10.330210+0000","last_undegraded":"2023-10-07T23:22:10.330210+0000","last_fullsized":"2023-10-07T23:22:10.330210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:47.722190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239890+0000","last_change":"2023-10-07T23:22:09.237476+0000","last_active":"2023-10-07T23:22:09.239890+0000","last_peered":"2023-10-07T23:22:09.239890+0000","last_clean":"2023-10-07T23:22:09.239890+0000","last_became_active":"2023-10-07T23:22:09.237143+0000","last_became_peered":"2023-10-07T23:22:09.237143+0000","last_unstale":"2023-10-07T23:22:09.239890+0000","last_undegraded":"2023-10-07T23:22:09.239890+0000","last_fullsized":"2023-10-07T23:22:09.239890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.223505+0000","last_change":"2023-10-07T23:22:08.198547+0000","last_active":"2023-10-07T23:22:09.223505+0000","last_peered":"2023-10-07T23:22:09.223505+0000","last_clean":"2023-10-07T23:22:09.223505+0000","last_became_active":"2023-10-07T23:22:08.197247+0000","last_became_peered":"2023-10-07T23:22:08.197247+0000","last_unstale":"2023-10-07T23:22:09.223505+0000","last_undegraded":"2023-10-07T23:22:09.223505+0000","last_fullsized":"2023-10-07T23:22:09.223505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:10.544875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.212509+0000","last_change":"2023-10-07T23:22:09.211298+0000","last_active":"2023-10-07T23:22:09.212509+0000","last_peered":"2023-10-07T23:22:09.212509+0000","last_clean":"2023-10-07T23:22:09.212509+0000","last_became_active":"2023-10-07T23:22:09.210873+0000","last_became_peered":"2023-10-07T23:22:09.210873+0000","last_unstale":"2023-10-07T23:22:09.212509+0000","last_undegraded":"2023-10-07T23:22:09.212509+0000","last_fullsized":"2023-10-07T23:22:09.212509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:10.463096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.203815+0000","last_change":"2023-10-07T23:22:08.196476+0000","last_active":"2023-10-07T23:22:09.203815+0000","last_peered":"2023-10-07T23:22:09.203815+0000","last_clean":"2023-10-07T23:22:09.203815+0000","last_became_active":"2023-10-07T23:22:08.196110+0000","last_became_peered":"2023-10-07T23:22:08.196110+0000","last_unstale":"2023-10-07T23:22:09.203815+0000","last_undegraded":"2023-10-07T23:22:09.203815+0000","last_fullsized":"2023-10-07T23:22:09.203815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.220542+0000","last_change":"2023-10-07T23:22:09.211891+0000","last_active":"2023-10-07T23:22:09.220542+0000","last_peered":"2023-10-07T23:22:09.220542+0000","last_clean":"2023-10-07T23:22:09.220542+0000","last_became_active":"2023-10-07T23:22:09.211518+0000","last_became_peered":"2023-10-07T23:22:09.211518+0000","last_unstale":"2023-10-07T23:22:09.220542+0000","last_undegraded":"2023-10-07T23:22:09.220542+0000","last_fullsized":"2023-10-07T23:22:09.220542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.222745+0000","last_change":"2023-10-07T23:22:08.198487+0000","last_active":"2023-10-07T23:22:09.222745+0000","last_peered":"2023-10-07T23:22:09.222745+0000","last_clean":"2023-10-07T23:22:09.222745+0000","last_became_active":"2023-10-07T23:22:08.198134+0000","last_became_peered":"2023-10-07T23:22:08.198134+0000","last_unstale":"2023-10-07T23:22:09.222745+0000","last_undegraded":"2023-10-07T23:22:09.222745+0000","last_fullsized":"2023-10-07T23:22:09.222745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:57.239785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248570+0000","last_change":"2023-10-07T23:22:09.243348+0000","last_active":"2023-10-07T23:22:09.248570+0000","last_peered":"2023-10-07T23:22:09.248570+0000","last_clean":"2023-10-07T23:22:09.248570+0000","last_became_active":"2023-10-07T23:22:09.240791+0000","last_became_peered":"2023-10-07T23:22:09.240791+0000","last_unstale":"2023-10-07T23:22:09.248570+0000","last_undegraded":"2023-10-07T23:22:09.248570+0000","last_fullsized":"2023-10-07T23:22:09.248570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.206124+0000","last_change":"2023-10-07T23:22:08.192121+0000","last_active":"2023-10-07T23:22:09.206124+0000","last_peered":"2023-10-07T23:22:09.206124+0000","last_clean":"2023-10-07T23:22:09.206124+0000","last_became_active":"2023-10-07T23:22:08.191858+0000","last_became_peered":"2023-10-07T23:22:08.191858+0000","last_unstale":"2023-10-07T23:22:09.206124+0000","last_undegraded":"2023-10-07T23:22:09.206124+0000","last_fullsized":"2023-10-07T23:22:09.206124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:39.776359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237366+0000","last_change":"2023-10-07T23:22:09.232607+0000","last_active":"2023-10-07T23:22:09.237366+0000","last_peered":"2023-10-07T23:22:09.237366+0000","last_clean":"2023-10-07T23:22:09.237366+0000","last_became_active":"2023-10-07T23:22:09.232244+0000","last_became_peered":"2023-10-07T23:22:09.232244+0000","last_unstale":"2023-10-07T23:22:09.237366+0000","last_undegraded":"2023-10-07T23:22:09.237366+0000","last_fullsized":"2023-10-07T23:22:09.237366+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:14.898343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.215549+0000","last_change":"2023-10-07T23:22:08.200328+0000","last_active":"2023-10-07T23:22:09.215549+0000","last_peered":"2023-10-07T23:22:09.215549+0000","last_clean":"2023-10-07T23:22:09.215549+0000","last_became_active":"2023-10-07T23:22:08.199980+0000","last_became_peered":"2023-10-07T23:22:08.199980+0000","last_unstale":"2023-10-07T23:22:09.215549+0000","last_undegraded":"2023-10-07T23:22:09.215549+0000","last_fullsized":"2023-10-07T23:22:09.215549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:40.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236833+0000","last_change":"2023-10-07T23:22:09.234281+0000","last_active":"2023-10-07T23:22:09.236833+0000","last_peered":"2023-10-07T23:22:09.236833+0000","last_clean":"2023-10-07T23:22:09.236833+0000","last_became_active":"2023-10-07T23:22:09.230674+0000","last_became_peered":"2023-10-07T23:22:09.230674+0000","last_unstale":"2023-10-07T23:22:09.236833+0000","last_undegraded":"2023-10-07T23:22:09.236833+0000","last_fullsized":"2023-10-07T23:22:09.236833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.208496+0000","last_change":"2023-10-07T23:22:08.193192+0000","last_active":"2023-10-07T23:22:09.208496+0000","last_peered":"2023-10-07T23:22:09.208496+0000","last_clean":"2023-10-07T23:22:09.208496+0000","last_became_active":"2023-10-07T23:22:08.192927+0000","last_became_peered":"2023-10-07T23:22:08.192927+0000","last_unstale":"2023-10-07T23:22:09.208496+0000","last_undegraded":"2023-10-07T23:22:09.208496+0000","last_fullsized":"2023-10-07T23:22:09.208496+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:39.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217208+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:22:09.217208+0000","last_peered":"2023-10-07T23:22:09.217208+0000","last_clean":"2023-10-07T23:22:09.217208+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:22:09.217208+0000","last_undegraded":"2023-10-07T23:22:09.217208+0000","last_fullsized":"2023-10-07T23:22:09.217208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073588,"statfs":{"total":96636764160,"available":96331354112,"internally_reserved":0,"allocated":987136,"data_stored":739452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":405504,"data_stored":150552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":393216,"data_stored":149064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93999999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.755}]},{"osd":3,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.0669999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.153}]},{"osd":6,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.095},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.1140000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.1950000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302344,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069496,"statfs":{"total":96636764160,"available":96327163904,"internally_reserved":0,"allocated":983040,"data_stored":739362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":405504,"data_stored":150552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:22:13.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:22:13.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 Processor -- start 2023-10-07T23:22:13.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 -- start start 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3dc0155f20 0x7f3dc0159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc0159e60 con 0x7f3dc00cbb20 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc015a140 con 0x7f3dc0155b50 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc7f5a700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc006c930 con 0x7f3dc0155f20 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc54f5700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3dc00cbb20 0x7f3dc006cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36240/0 (socket says 172.21.15.57:36240) 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc54f5700 1 -- 172.21.15.57:0/2865683590 learned_addr learned my addr 172.21.15.57:0/2865683590 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:13.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.341+0000 7f3dc64f7700 1 --2- 172.21.15.57:0/2865683590 >> v2:172.21.15.154:3300/0 conn(0x7f3dc0155f20 0x7f3dc0159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc4cf4700 1 -- 172.21.15.57:0/2865683590 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102763146 0 0) 0x7f3dc0159e60 con 0x7f3dc00cbb20 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc4cf4700 1 -- 172.21.15.57:0/2865683590 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dac003570 con 0x7f3dc00cbb20 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc64f7700 1 -- 172.21.15.57:0/2865683590 >> v1:172.21.15.154:6789/0 conn(0x7f3dc0155b50 legacy=0x7f3dc00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc64f7700 1 -- 172.21.15.57:0/2865683590 >> v1:172.21.15.57:6789/0 conn(0x7f3dc00cbb20 legacy=0x7f3dc006cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc64f7700 1 -- 172.21.15.57:0/2865683590 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc015a420 con 0x7f3dc0155f20 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc64f7700 1 --2- 172.21.15.57:0/2865683590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155f20 0x7f3dc0159880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009e60 tx=0x7f3db8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ff9a3ddfd637f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc4cf4700 1 -- 172.21.15.57:0/2865683590 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db800e070 con 0x7f3dc0155f20 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc4cf4700 1 -- 172.21.15.57:0/2865683590 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db800bb60 con 0x7f3dc0155f20 2023-10-07T23:22:13.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc4cf4700 1 -- 172.21.15.57:0/2865683590 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8012430 con 0x7f3dc0155f20 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/2865683590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155f20 msgr2=0x7f3dc0159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/2865683590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155f20 0x7f3dc0159880 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009e60 tx=0x7f3db8009780 comp rx=0 tx=0).stop 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/2865683590 shutdown_connections 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/2865683590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155f20 0x7f3dc0159880 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/2865683590 >> 172.21.15.57:0/2865683590 conn(0x7f3dc00cce30 msgr2=0x7f3dc00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/2865683590 shutdown_connections 2023-10-07T23:22:13.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.345+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/2865683590 wait complete. 2023-10-07T23:22:13.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 Processor -- start 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 -- start start 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dc00cbb20 0x7f3dc0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155b50 0x7f3dc016e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc01523d0 con 0x7f3dc0155f20 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0152650 con 0x7f3dc0155b50 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc7f5a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0152930 con 0x7f3dc00cbb20 2023-10-07T23:22:13.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc64f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc64f7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56812/0 (socket says 172.21.15.57:56812) 2023-10-07T23:22:13.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc64f7700 1 -- 172.21.15.57:0/518154193 learned_addr learned my addr 172.21.15.57:0/518154193 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:13.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc54f5700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155b50 0x7f3dc016e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.349+0000 7f3dc5cf6700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dc00cbb20 0x7f3dc0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 -- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dc00cbb20 msgr2=0x7f3dc0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dc00cbb20 0x7f3dc0151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 -- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155b50 msgr2=0x7f3dc016e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155b50 0x7f3dc016e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 -- 172.21.15.57:0/518154193 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8009a20 con 0x7f3dc0155f20 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc64f7700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00c3160 tx=0x7f3db8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8012d20 con 0x7f3dc0155f20 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8012e80 con 0x7f3dc0155f20 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0152c10 con 0x7f3dc0155f20 2023-10-07T23:22:13.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8025690 con 0x7f3dc0155f20 2023-10-07T23:22:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc016d770 con 0x7f3dc0155f20 2023-10-07T23:22:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc00711c0 con 0x7f3dc0155f20 2023-10-07T23:22:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3db8003050 con 0x7f3dc0155f20 2023-10-07T23:22:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 0x7f3da4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3db807e3f0 con 0x7f3dc0155f20 2023-10-07T23:22:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.353+0000 7f3dc5cf6700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 0x7f3da4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.357+0000 7f3dc5cf6700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 0x7f3da4062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc003a80 tx=0x7f3dbc007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.365+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3db804e9e0 con 0x7f3dc0155f20 2023-10-07T23:22:13.527 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.521+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dc015a700 con 0x7f3da4060550 2023-10-07T23:22:13.546 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.541+0000 7f3db5ffb700 1 -- 172.21.15.57:0/518154193 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+267346 (secure 0 0 0) 0x7f3dc015a700 con 0x7f3da4060550 2023-10-07T23:22:13.549 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:13.551 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 msgr2=0x7f3da4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 0x7f3da4062a00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc003a80 tx=0x7f3dbc007ec0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 msgr2=0x7f3dc014dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00c3160 tx=0x7f3db8019000 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 shutdown_connections 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dc0155f20 0x7f3dc014dd50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3da4060550 0x7f3da4062a00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dc0155b50 0x7f3dc016e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 --2- 172.21.15.57:0/518154193 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dc00cbb20 0x7f3dc0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 >> 172.21.15.57:0/518154193 conn(0x7f3dc00cce30 msgr2=0x7f3dc00c1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 shutdown_connections 2023-10-07T23:22:13.552 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.545+0000 7f3dc7f5a700 1 -- 172.21.15.57:0/518154193 wait complete. 2023-10-07T23:22:13.553 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:22:13.565 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2023-10-07T23:22:13.086755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398952,"kb_used_data":4264,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575768,"statfs":{"total":773094113280,"available":770637586432,"internally_reserved":0,"allocated":4366336,"data_stored":2377300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.917060"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.230521+0000","last_change":"2023-10-07T23:22:09.227740+0000","last_active":"2023-10-07T23:22:09.230521+0000","last_peered":"2023-10-07T23:22:09.230521+0000","last_clean":"2023-10-07T23:22:09.230521+0000","last_became_active":"2023-10-07T23:22:09.227359+0000","last_became_peered":"2023-10-07T23:22:09.227359+0000","last_unstale":"2023-10-07T23:22:09.230521+0000","last_undegraded":"2023-10-07T23:22:09.230521+0000","last_fullsized":"2023-10-07T23:22:09.230521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247996+0000","last_change":"2023-10-07T23:22:09.240862+0000","last_active":"2023-10-07T23:22:09.247996+0000","last_peered":"2023-10-07T23:22:09.247996+0000","last_clean":"2023-10-07T23:22:09.247996+0000","last_became_active":"2023-10-07T23:22:09.240353+0000","last_became_peered":"2023-10-07T23:22:09.240353+0000","last_unstale":"2023-10-07T23:22:09.247996+0000","last_undegraded":"2023-10-07T23:22:09.247996+0000","last_fullsized":"2023-10-07T23:22:09.247996+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248424+0000","last_change":"2023-10-07T23:22:09.243291+0000","last_active":"2023-10-07T23:22:09.248424+0000","last_peered":"2023-10-07T23:22:09.248424+0000","last_clean":"2023-10-07T23:22:09.248424+0000","last_became_active":"2023-10-07T23:22:09.242972+0000","last_became_peered":"2023-10-07T23:22:09.242972+0000","last_unstale":"2023-10-07T23:22:09.248424+0000","last_undegraded":"2023-10-07T23:22:09.248424+0000","last_fullsized":"2023-10-07T23:22:09.248424+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244674+0000","last_change":"2023-10-07T23:22:09.235853+0000","last_active":"2023-10-07T23:22:09.244674+0000","last_peered":"2023-10-07T23:22:09.244674+0000","last_clean":"2023-10-07T23:22:09.244674+0000","last_became_active":"2023-10-07T23:22:09.235686+0000","last_became_peered":"2023-10-07T23:22:09.235686+0000","last_unstale":"2023-10-07T23:22:09.244674+0000","last_undegraded":"2023-10-07T23:22:09.244674+0000","last_fullsized":"2023-10-07T23:22:09.244674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:51.866516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238240+0000","last_change":"2023-10-07T23:22:09.231694+0000","last_active":"2023-10-07T23:22:09.238240+0000","last_peered":"2023-10-07T23:22:09.238240+0000","last_clean":"2023-10-07T23:22:09.238240+0000","last_became_active":"2023-10-07T23:22:09.231199+0000","last_became_peered":"2023-10-07T23:22:09.231199+0000","last_unstale":"2023-10-07T23:22:09.238240+0000","last_undegraded":"2023-10-07T23:22:09.238240+0000","last_fullsized":"2023-10-07T23:22:09.238240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:04.106129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239600+0000","last_change":"2023-10-07T23:22:09.230322+0000","last_active":"2023-10-07T23:22:09.239600+0000","last_peered":"2023-10-07T23:22:09.239600+0000","last_clean":"2023-10-07T23:22:09.239600+0000","last_became_active":"2023-10-07T23:22:09.229441+0000","last_became_peered":"2023-10-07T23:22:09.229441+0000","last_unstale":"2023-10-07T23:22:09.239600+0000","last_undegraded":"2023-10-07T23:22:09.239600+0000","last_fullsized":"2023-10-07T23:22:09.239600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:22.309792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235790+0000","last_change":"2023-10-07T23:22:09.233463+0000","last_active":"2023-10-07T23:22:09.235790+0000","last_peered":"2023-10-07T23:22:09.235790+0000","last_clean":"2023-10-07T23:22:09.235790+0000","last_became_active":"2023-10-07T23:22:09.230666+0000","last_became_peered":"2023-10-07T23:22:09.230666+0000","last_unstale":"2023-10-07T23:22:09.235790+0000","last_undegraded":"2023-10-07T23:22:09.235790+0000","last_fullsized":"2023-10-07T23:22:09.235790+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231066+0000","last_change":"2023-10-07T23:22:09.224898+0000","last_active":"2023-10-07T23:22:09.231066+0000","last_peered":"2023-10-07T23:22:09.231066+0000","last_clean":"2023-10-07T23:22:09.231066+0000","last_became_active":"2023-10-07T23:22:09.224592+0000","last_became_peered":"2023-10-07T23:22:09.224592+0000","last_unstale":"2023-10-07T23:22:09.231066+0000","last_undegraded":"2023-10-07T23:22:09.231066+0000","last_fullsized":"2023-10-07T23:22:09.231066+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.242659+0000","last_change":"2023-10-07T23:22:09.237812+0000","last_active":"2023-10-07T23:22:09.242659+0000","last_peered":"2023-10-07T23:22:09.242659+0000","last_clean":"2023-10-07T23:22:09.242659+0000","last_became_active":"2023-10-07T23:22:09.237475+0000","last_became_peered":"2023-10-07T23:22:09.237475+0000","last_unstale":"2023-10-07T23:22:09.242659+0000","last_undegraded":"2023-10-07T23:22:09.242659+0000","last_fullsized":"2023-10-07T23:22:09.242659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:49.173670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244670+0000","last_change":"2023-10-07T23:22:09.236931+0000","last_active":"2023-10-07T23:22:09.244670+0000","last_peered":"2023-10-07T23:22:09.244670+0000","last_clean":"2023-10-07T23:22:09.244670+0000","last_became_active":"2023-10-07T23:22:09.236529+0000","last_became_peered":"2023-10-07T23:22:09.236529+0000","last_unstale":"2023-10-07T23:22:09.244670+0000","last_undegraded":"2023-10-07T23:22:09.244670+0000","last_fullsized":"2023-10-07T23:22:09.244670+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:53:50.386927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248392+0000","last_change":"2023-10-07T23:22:09.243084+0000","last_active":"2023-10-07T23:22:09.248392+0000","last_peered":"2023-10-07T23:22:09.248392+0000","last_clean":"2023-10-07T23:22:09.248392+0000","last_became_active":"2023-10-07T23:22:09.242731+0000","last_became_peered":"2023-10-07T23:22:09.242731+0000","last_unstale":"2023-10-07T23:22:09.248392+0000","last_undegraded":"2023-10-07T23:22:09.248392+0000","last_fullsized":"2023-10-07T23:22:09.248392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:35.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.242813+0000","last_change":"2023-10-07T23:22:09.242028+0000","last_active":"2023-10-07T23:22:09.242813+0000","last_peered":"2023-10-07T23:22:09.242813+0000","last_clean":"2023-10-07T23:22:09.242813+0000","last_became_active":"2023-10-07T23:22:09.241684+0000","last_became_peered":"2023-10-07T23:22:09.241684+0000","last_unstale":"2023-10-07T23:22:09.242813+0000","last_undegraded":"2023-10-07T23:22:09.242813+0000","last_fullsized":"2023-10-07T23:22:09.242813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:29.967352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243496+0000","last_change":"2023-10-07T23:22:09.242444+0000","last_active":"2023-10-07T23:22:09.243496+0000","last_peered":"2023-10-07T23:22:09.243496+0000","last_clean":"2023-10-07T23:22:09.243496+0000","last_became_active":"2023-10-07T23:22:09.242008+0000","last_became_peered":"2023-10-07T23:22:09.242008+0000","last_unstale":"2023-10-07T23:22:09.243496+0000","last_undegraded":"2023-10-07T23:22:09.243496+0000","last_fullsized":"2023-10-07T23:22:09.243496+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:10.135861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244168+0000","last_change":"2023-10-07T23:22:09.234930+0000","last_active":"2023-10-07T23:22:09.244168+0000","last_peered":"2023-10-07T23:22:09.244168+0000","last_clean":"2023-10-07T23:22:09.244168+0000","last_became_active":"2023-10-07T23:22:09.229596+0000","last_became_peered":"2023-10-07T23:22:09.229596+0000","last_unstale":"2023-10-07T23:22:09.244168+0000","last_undegraded":"2023-10-07T23:22:09.244168+0000","last_fullsized":"2023-10-07T23:22:09.244168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:37.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.230258+0000","last_change":"2023-10-07T23:22:09.222600+0000","last_active":"2023-10-07T23:22:09.230258+0000","last_peered":"2023-10-07T23:22:09.230258+0000","last_clean":"2023-10-07T23:22:09.230258+0000","last_became_active":"2023-10-07T23:22:09.222314+0000","last_became_peered":"2023-10-07T23:22:09.222314+0000","last_unstale":"2023-10-07T23:22:09.230258+0000","last_undegraded":"2023-10-07T23:22:09.230258+0000","last_fullsized":"2023-10-07T23:22:09.230258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:35.637074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248418+0000","last_change":"2023-10-07T23:22:09.243142+0000","last_active":"2023-10-07T23:22:09.248418+0000","last_peered":"2023-10-07T23:22:09.248418+0000","last_clean":"2023-10-07T23:22:09.248418+0000","last_became_active":"2023-10-07T23:22:09.242789+0000","last_became_peered":"2023-10-07T23:22:09.242789+0000","last_unstale":"2023-10-07T23:22:09.248418+0000","last_undegraded":"2023-10-07T23:22:09.248418+0000","last_fullsized":"2023-10-07T23:22:09.248418+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235636+0000","last_change":"2023-10-07T23:22:09.230550+0000","last_active":"2023-10-07T23:22:09.235636+0000","last_peered":"2023-10-07T23:22:09.235636+0000","last_clean":"2023-10-07T23:22:09.235636+0000","last_became_active":"2023-10-07T23:22:09.227637+0000","last_became_peered":"2023-10-07T23:22:09.227637+0000","last_unstale":"2023-10-07T23:22:09.235636+0000","last_undegraded":"2023-10-07T23:22:09.235636+0000","last_fullsized":"2023-10-07T23:22:09.235636+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:29.967352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247730+0000","last_change":"2023-10-07T23:22:09.240045+0000","last_active":"2023-10-07T23:22:09.247730+0000","last_peered":"2023-10-07T23:22:09.247730+0000","last_clean":"2023-10-07T23:22:09.247730+0000","last_became_active":"2023-10-07T23:22:09.236783+0000","last_became_peered":"2023-10-07T23:22:09.236783+0000","last_unstale":"2023-10-07T23:22:09.247730+0000","last_undegraded":"2023-10-07T23:22:09.247730+0000","last_fullsized":"2023-10-07T23:22:09.247730+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:40.100405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248313+0000","last_change":"2023-10-07T23:22:09.242606+0000","last_active":"2023-10-07T23:22:09.248313+0000","last_peered":"2023-10-07T23:22:09.248313+0000","last_clean":"2023-10-07T23:22:09.248313+0000","last_became_active":"2023-10-07T23:22:09.241969+0000","last_became_peered":"2023-10-07T23:22:09.241969+0000","last_unstale":"2023-10-07T23:22:09.248313+0000","last_undegraded":"2023-10-07T23:22:09.248313+0000","last_fullsized":"2023-10-07T23:22:09.248313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:19:17.259019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.228809+0000","last_change":"2023-10-07T23:22:09.227795+0000","last_active":"2023-10-07T23:22:09.228809+0000","last_peered":"2023-10-07T23:22:09.228809+0000","last_clean":"2023-10-07T23:22:09.228809+0000","last_became_active":"2023-10-07T23:22:09.227335+0000","last_became_peered":"2023-10-07T23:22:09.227335+0000","last_unstale":"2023-10-07T23:22:09.228809+0000","last_undegraded":"2023-10-07T23:22:09.228809+0000","last_fullsized":"2023-10-07T23:22:09.228809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:10.401035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.213145+0000","last_change":"2023-10-07T23:22:09.211146+0000","last_active":"2023-10-07T23:22:09.213145+0000","last_peered":"2023-10-07T23:22:09.213145+0000","last_clean":"2023-10-07T23:22:09.213145+0000","last_became_active":"2023-10-07T23:22:09.210702+0000","last_became_peered":"2023-10-07T23:22:09.210702+0000","last_unstale":"2023-10-07T23:22:09.213145+0000","last_undegraded":"2023-10-07T23:22:09.213145+0000","last_fullsized":"2023-10-07T23:22:09.213145+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243046+0000","last_change":"2023-10-07T23:22:09.238893+0000","last_active":"2023-10-07T23:22:09.243046+0000","last_peered":"2023-10-07T23:22:09.243046+0000","last_clean":"2023-10-07T23:22:09.243046+0000","last_became_active":"2023-10-07T23:22:09.238549+0000","last_became_peered":"2023-10-07T23:22:09.238549+0000","last_unstale":"2023-10-07T23:22:09.243046+0000","last_undegraded":"2023-10-07T23:22:09.243046+0000","last_fullsized":"2023-10-07T23:22:09.243046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:43:18.264986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.229062+0000","last_change":"2023-10-07T23:22:09.224304+0000","last_active":"2023-10-07T23:22:09.229062+0000","last_peered":"2023-10-07T23:22:09.229062+0000","last_clean":"2023-10-07T23:22:09.229062+0000","last_became_active":"2023-10-07T23:22:09.223992+0000","last_became_peered":"2023-10-07T23:22:09.223992+0000","last_unstale":"2023-10-07T23:22:09.229062+0000","last_undegraded":"2023-10-07T23:22:09.229062+0000","last_fullsized":"2023-10-07T23:22:09.229062+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:44.258733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232065+0000","last_change":"2023-10-07T23:22:09.225439+0000","last_active":"2023-10-07T23:22:09.232065+0000","last_peered":"2023-10-07T23:22:09.232065+0000","last_clean":"2023-10-07T23:22:09.232065+0000","last_became_active":"2023-10-07T23:22:09.225050+0000","last_became_peered":"2023-10-07T23:22:09.225050+0000","last_unstale":"2023-10-07T23:22:09.232065+0000","last_undegraded":"2023-10-07T23:22:09.232065+0000","last_fullsized":"2023-10-07T23:22:09.232065+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:59.536276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244494+0000","last_change":"2023-10-07T23:22:09.236386+0000","last_active":"2023-10-07T23:22:09.244494+0000","last_peered":"2023-10-07T23:22:09.244494+0000","last_clean":"2023-10-07T23:22:09.244494+0000","last_became_active":"2023-10-07T23:22:09.236100+0000","last_became_peered":"2023-10-07T23:22:09.236100+0000","last_unstale":"2023-10-07T23:22:09.244494+0000","last_undegraded":"2023-10-07T23:22:09.244494+0000","last_fullsized":"2023-10-07T23:22:09.244494+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:52:27.578614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245103+0000","last_change":"2023-10-07T23:22:09.230696+0000","last_active":"2023-10-07T23:22:09.245103+0000","last_peered":"2023-10-07T23:22:09.245103+0000","last_clean":"2023-10-07T23:22:09.245103+0000","last_became_active":"2023-10-07T23:22:09.230360+0000","last_became_peered":"2023-10-07T23:22:09.230360+0000","last_unstale":"2023-10-07T23:22:09.245103+0000","last_undegraded":"2023-10-07T23:22:09.245103+0000","last_fullsized":"2023-10-07T23:22:09.245103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:13.198994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245064+0000","last_change":"2023-10-07T23:22:09.237673+0000","last_active":"2023-10-07T23:22:09.245064+0000","last_peered":"2023-10-07T23:22:09.245064+0000","last_clean":"2023-10-07T23:22:09.245064+0000","last_became_active":"2023-10-07T23:22:09.236679+0000","last_became_peered":"2023-10-07T23:22:09.236679+0000","last_unstale":"2023-10-07T23:22:09.245064+0000","last_undegraded":"2023-10-07T23:22:09.245064+0000","last_fullsized":"2023-10-07T23:22:09.245064+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:10.075644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232268+0000","last_change":"2023-10-07T23:22:09.225102+0000","last_active":"2023-10-07T23:22:09.232268+0000","last_peered":"2023-10-07T23:22:09.232268+0000","last_clean":"2023-10-07T23:22:09.232268+0000","last_became_active":"2023-10-07T23:22:09.224760+0000","last_became_peered":"2023-10-07T23:22:09.224760+0000","last_unstale":"2023-10-07T23:22:09.232268+0000","last_undegraded":"2023-10-07T23:22:09.232268+0000","last_fullsized":"2023-10-07T23:22:09.232268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:40.100405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237386+0000","last_change":"2023-10-07T23:22:09.232606+0000","last_active":"2023-10-07T23:22:09.237386+0000","last_peered":"2023-10-07T23:22:09.237386+0000","last_clean":"2023-10-07T23:22:09.237386+0000","last_became_active":"2023-10-07T23:22:09.232274+0000","last_became_peered":"2023-10-07T23:22:09.232274+0000","last_unstale":"2023-10-07T23:22:09.237386+0000","last_undegraded":"2023-10-07T23:22:09.237386+0000","last_fullsized":"2023-10-07T23:22:09.237386+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:16:50.522855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.234693+0000","last_change":"2023-10-07T23:22:09.229905+0000","last_active":"2023-10-07T23:22:09.234693+0000","last_peered":"2023-10-07T23:22:09.234693+0000","last_clean":"2023-10-07T23:22:09.234693+0000","last_became_active":"2023-10-07T23:22:09.227574+0000","last_became_peered":"2023-10-07T23:22:09.227574+0000","last_unstale":"2023-10-07T23:22:09.234693+0000","last_undegraded":"2023-10-07T23:22:09.234693+0000","last_fullsized":"2023-10-07T23:22:09.234693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:40:02.590917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.226692+0000","last_change":"2023-10-07T23:22:09.223398+0000","last_active":"2023-10-07T23:22:09.226692+0000","last_peered":"2023-10-07T23:22:09.226692+0000","last_clean":"2023-10-07T23:22:09.226692+0000","last_became_active":"2023-10-07T23:22:09.222975+0000","last_became_peered":"2023-10-07T23:22:09.222975+0000","last_unstale":"2023-10-07T23:22:09.226692+0000","last_undegraded":"2023-10-07T23:22:09.226692+0000","last_fullsized":"2023-10-07T23:22:09.226692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231085+0000","last_change":"2023-10-07T23:22:09.225723+0000","last_active":"2023-10-07T23:22:09.231085+0000","last_peered":"2023-10-07T23:22:09.231085+0000","last_clean":"2023-10-07T23:22:09.231085+0000","last_became_active":"2023-10-07T23:22:09.225153+0000","last_became_peered":"2023-10-07T23:22:09.225153+0000","last_unstale":"2023-10-07T23:22:09.231085+0000","last_undegraded":"2023-10-07T23:22:09.231085+0000","last_fullsized":"2023-10-07T23:22:09.231085+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:33:14.039148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.228763+0000","last_change":"2023-10-07T23:22:09.224278+0000","last_active":"2023-10-07T23:22:09.228763+0000","last_peered":"2023-10-07T23:22:09.228763+0000","last_clean":"2023-10-07T23:22:09.228763+0000","last_became_active":"2023-10-07T23:22:09.223924+0000","last_became_peered":"2023-10-07T23:22:09.223924+0000","last_unstale":"2023-10-07T23:22:09.228763+0000","last_undegraded":"2023-10-07T23:22:09.228763+0000","last_fullsized":"2023-10-07T23:22:09.228763+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.234482+0000","last_change":"2023-10-07T23:22:08.196666+0000","last_active":"2023-10-07T23:22:09.234482+0000","last_peered":"2023-10-07T23:22:09.234482+0000","last_clean":"2023-10-07T23:22:09.234482+0000","last_became_active":"2023-10-07T23:22:08.196325+0000","last_became_peered":"2023-10-07T23:22:08.196325+0000","last_unstale":"2023-10-07T23:22:09.234482+0000","last_undegraded":"2023-10-07T23:22:09.234482+0000","last_fullsized":"2023-10-07T23:22:09.234482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:24.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.211766+0000","last_change":"2023-10-07T23:22:09.210142+0000","last_active":"2023-10-07T23:22:09.211766+0000","last_peered":"2023-10-07T23:22:09.211766+0000","last_clean":"2023-10-07T23:22:09.211766+0000","last_became_active":"2023-10-07T23:22:09.209843+0000","last_became_peered":"2023-10-07T23:22:09.209843+0000","last_unstale":"2023-10-07T23:22:09.211766+0000","last_undegraded":"2023-10-07T23:22:09.211766+0000","last_fullsized":"2023-10-07T23:22:09.211766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:04.648785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236175+0000","last_change":"2023-10-07T23:22:08.199116+0000","last_active":"2023-10-07T23:22:09.236175+0000","last_peered":"2023-10-07T23:22:09.236175+0000","last_clean":"2023-10-07T23:22:09.236175+0000","last_became_active":"2023-10-07T23:22:08.198514+0000","last_became_peered":"2023-10-07T23:22:08.198514+0000","last_unstale":"2023-10-07T23:22:09.236175+0000","last_undegraded":"2023-10-07T23:22:09.236175+0000","last_fullsized":"2023-10-07T23:22:09.236175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:09.387655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245572+0000","last_change":"2023-10-07T23:22:09.241884+0000","last_active":"2023-10-07T23:22:09.245572+0000","last_peered":"2023-10-07T23:22:09.245572+0000","last_clean":"2023-10-07T23:22:09.245572+0000","last_became_active":"2023-10-07T23:22:09.241629+0000","last_became_peered":"2023-10-07T23:22:09.241629+0000","last_unstale":"2023-10-07T23:22:09.245572+0000","last_undegraded":"2023-10-07T23:22:09.245572+0000","last_fullsized":"2023-10-07T23:22:09.245572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:44.258733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245333+0000","last_change":"2023-10-07T23:22:08.197599+0000","last_active":"2023-10-07T23:22:09.245333+0000","last_peered":"2023-10-07T23:22:09.245333+0000","last_clean":"2023-10-07T23:22:09.245333+0000","last_became_active":"2023-10-07T23:22:08.197276+0000","last_became_peered":"2023-10-07T23:22:08.197276+0000","last_unstale":"2023-10-07T23:22:09.245333+0000","last_undegraded":"2023-10-07T23:22:09.245333+0000","last_fullsized":"2023-10-07T23:22:09.245333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:43.245354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244594+0000","last_change":"2023-10-07T23:22:09.236900+0000","last_active":"2023-10-07T23:22:09.244594+0000","last_peered":"2023-10-07T23:22:09.244594+0000","last_clean":"2023-10-07T23:22:09.244594+0000","last_became_active":"2023-10-07T23:22:09.236202+0000","last_became_peered":"2023-10-07T23:22:09.236202+0000","last_unstale":"2023-10-07T23:22:09.244594+0000","last_undegraded":"2023-10-07T23:22:09.244594+0000","last_fullsized":"2023-10-07T23:22:09.244594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:34:11.022020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.244641+0000","last_change":"2023-10-07T23:22:08.200777+0000","last_active":"2023-10-07T23:22:09.244641+0000","last_peered":"2023-10-07T23:22:09.244641+0000","last_clean":"2023-10-07T23:22:09.244641+0000","last_became_active":"2023-10-07T23:22:08.200508+0000","last_became_peered":"2023-10-07T23:22:08.200508+0000","last_unstale":"2023-10-07T23:22:09.244641+0000","last_undegraded":"2023-10-07T23:22:09.244641+0000","last_fullsized":"2023-10-07T23:22:09.244641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:39.410643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235598+0000","last_change":"2023-10-07T23:22:09.230430+0000","last_active":"2023-10-07T23:22:09.235598+0000","last_peered":"2023-10-07T23:22:09.235598+0000","last_clean":"2023-10-07T23:22:09.235598+0000","last_became_active":"2023-10-07T23:22:09.230173+0000","last_became_peered":"2023-10-07T23:22:09.230173+0000","last_unstale":"2023-10-07T23:22:09.235598+0000","last_undegraded":"2023-10-07T23:22:09.235598+0000","last_fullsized":"2023-10-07T23:22:09.235598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:22.025077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.246332+0000","last_change":"2023-10-07T23:22:08.195534+0000","last_active":"2023-10-07T23:22:09.246332+0000","last_peered":"2023-10-07T23:22:09.246332+0000","last_clean":"2023-10-07T23:22:09.246332+0000","last_became_active":"2023-10-07T23:22:08.193314+0000","last_became_peered":"2023-10-07T23:22:08.193314+0000","last_unstale":"2023-10-07T23:22:09.246332+0000","last_undegraded":"2023-10-07T23:22:09.246332+0000","last_fullsized":"2023-10-07T23:22:09.246332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:24.636788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232228+0000","last_change":"2023-10-07T23:22:09.226598+0000","last_active":"2023-10-07T23:22:09.232228+0000","last_peered":"2023-10-07T23:22:09.232228+0000","last_clean":"2023-10-07T23:22:09.232228+0000","last_became_active":"2023-10-07T23:22:09.226207+0000","last_became_peered":"2023-10-07T23:22:09.226207+0000","last_unstale":"2023-10-07T23:22:09.232228+0000","last_undegraded":"2023-10-07T23:22:09.232228+0000","last_fullsized":"2023-10-07T23:22:09.232228+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:20.074931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245464+0000","last_change":"2023-10-07T23:22:08.193388+0000","last_active":"2023-10-07T23:22:09.245464+0000","last_peered":"2023-10-07T23:22:09.245464+0000","last_clean":"2023-10-07T23:22:09.245464+0000","last_became_active":"2023-10-07T23:22:08.193083+0000","last_became_peered":"2023-10-07T23:22:08.193083+0000","last_unstale":"2023-10-07T23:22:09.245464+0000","last_undegraded":"2023-10-07T23:22:09.245464+0000","last_fullsized":"2023-10-07T23:22:09.245464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:41.116405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.212184+0000","last_change":"2023-10-07T23:22:09.210142+0000","last_active":"2023-10-07T23:22:09.212184+0000","last_peered":"2023-10-07T23:22:09.212184+0000","last_clean":"2023-10-07T23:22:09.212184+0000","last_became_active":"2023-10-07T23:22:09.209698+0000","last_became_peered":"2023-10-07T23:22:09.209698+0000","last_unstale":"2023-10-07T23:22:09.212184+0000","last_undegraded":"2023-10-07T23:22:09.212184+0000","last_fullsized":"2023-10-07T23:22:09.212184+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:20.763682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217700+0000","last_change":"2023-10-07T23:22:08.196651+0000","last_active":"2023-10-07T23:22:09.217700+0000","last_peered":"2023-10-07T23:22:09.217700+0000","last_clean":"2023-10-07T23:22:09.217700+0000","last_became_active":"2023-10-07T23:22:08.196246+0000","last_became_peered":"2023-10-07T23:22:08.196246+0000","last_unstale":"2023-10-07T23:22:09.217700+0000","last_undegraded":"2023-10-07T23:22:09.217700+0000","last_fullsized":"2023-10-07T23:22:09.217700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:05.186215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.241112+0000","last_change":"2023-10-07T23:22:09.229068+0000","last_active":"2023-10-07T23:22:09.241112+0000","last_peered":"2023-10-07T23:22:09.241112+0000","last_clean":"2023-10-07T23:22:09.241112+0000","last_became_active":"2023-10-07T23:22:09.228732+0000","last_became_peered":"2023-10-07T23:22:09.228732+0000","last_unstale":"2023-10-07T23:22:09.241112+0000","last_undegraded":"2023-10-07T23:22:09.241112+0000","last_fullsized":"2023-10-07T23:22:09.241112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:49:53.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.205608+0000","last_change":"2023-10-07T23:22:08.189598+0000","last_active":"2023-10-07T23:22:09.205608+0000","last_peered":"2023-10-07T23:22:09.205608+0000","last_clean":"2023-10-07T23:22:09.205608+0000","last_became_active":"2023-10-07T23:22:08.189266+0000","last_became_peered":"2023-10-07T23:22:08.189266+0000","last_unstale":"2023-10-07T23:22:09.205608+0000","last_undegraded":"2023-10-07T23:22:09.205608+0000","last_fullsized":"2023-10-07T23:22:09.205608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:34:10.974537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238833+0000","last_change":"2023-10-07T23:22:09.237925+0000","last_active":"2023-10-07T23:22:09.238833+0000","last_peered":"2023-10-07T23:22:09.238833+0000","last_clean":"2023-10-07T23:22:09.238833+0000","last_became_active":"2023-10-07T23:22:09.237618+0000","last_became_peered":"2023-10-07T23:22:09.237618+0000","last_unstale":"2023-10-07T23:22:09.238833+0000","last_undegraded":"2023-10-07T23:22:09.238833+0000","last_fullsized":"2023-10-07T23:22:09.238833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.218106+0000","last_change":"2023-10-07T23:22:08.197437+0000","last_active":"2023-10-07T23:22:09.218106+0000","last_peered":"2023-10-07T23:22:09.218106+0000","last_clean":"2023-10-07T23:22:09.218106+0000","last_became_active":"2023-10-07T23:22:08.197130+0000","last_became_peered":"2023-10-07T23:22:08.197130+0000","last_unstale":"2023-10-07T23:22:09.218106+0000","last_undegraded":"2023-10-07T23:22:09.218106+0000","last_fullsized":"2023-10-07T23:22:09.218106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:03.635405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248272+0000","last_change":"2023-10-07T23:22:09.242371+0000","last_active":"2023-10-07T23:22:09.248272+0000","last_peered":"2023-10-07T23:22:09.248272+0000","last_clean":"2023-10-07T23:22:09.248272+0000","last_became_active":"2023-10-07T23:22:09.241987+0000","last_became_peered":"2023-10-07T23:22:09.241987+0000","last_unstale":"2023-10-07T23:22:09.248272+0000","last_undegraded":"2023-10-07T23:22:09.248272+0000","last_fullsized":"2023-10-07T23:22:09.248272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:17.860911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.204570+0000","last_change":"2023-10-07T23:22:08.192642+0000","last_active":"2023-10-07T23:22:09.204570+0000","last_peered":"2023-10-07T23:22:09.204570+0000","last_clean":"2023-10-07T23:22:09.204570+0000","last_became_active":"2023-10-07T23:22:08.192294+0000","last_became_peered":"2023-10-07T23:22:08.192294+0000","last_unstale":"2023-10-07T23:22:09.204570+0000","last_undegraded":"2023-10-07T23:22:09.204570+0000","last_fullsized":"2023-10-07T23:22:09.204570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:09.122481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247772+0000","last_change":"2023-10-07T23:22:09.240062+0000","last_active":"2023-10-07T23:22:09.247772+0000","last_peered":"2023-10-07T23:22:09.247772+0000","last_clean":"2023-10-07T23:22:09.247772+0000","last_became_active":"2023-10-07T23:22:09.239690+0000","last_became_peered":"2023-10-07T23:22:09.239690+0000","last_unstale":"2023-10-07T23:22:09.247772+0000","last_undegraded":"2023-10-07T23:22:09.247772+0000","last_fullsized":"2023-10-07T23:22:09.247772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:20.669822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237132+0000","last_change":"2023-10-07T23:22:08.199948+0000","last_active":"2023-10-07T23:22:09.237132+0000","last_peered":"2023-10-07T23:22:09.237132+0000","last_clean":"2023-10-07T23:22:09.237132+0000","last_became_active":"2023-10-07T23:22:08.199679+0000","last_became_peered":"2023-10-07T23:22:08.199679+0000","last_unstale":"2023-10-07T23:22:09.237132+0000","last_undegraded":"2023-10-07T23:22:09.237132+0000","last_fullsized":"2023-10-07T23:22:09.237132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:40.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.251381+0000","last_change":"2023-10-07T23:22:09.246678+0000","last_active":"2023-10-07T23:22:09.251381+0000","last_peered":"2023-10-07T23:22:09.251381+0000","last_clean":"2023-10-07T23:22:09.251381+0000","last_became_active":"2023-10-07T23:22:09.246396+0000","last_became_peered":"2023-10-07T23:22:09.246396+0000","last_unstale":"2023-10-07T23:22:09.251381+0000","last_undegraded":"2023-10-07T23:22:09.251381+0000","last_fullsized":"2023-10-07T23:22:09.251381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.250603+0000","last_change":"2023-10-07T23:22:08.197478+0000","last_active":"2023-10-07T23:22:09.250603+0000","last_peered":"2023-10-07T23:22:09.250603+0000","last_clean":"2023-10-07T23:22:09.250603+0000","last_became_active":"2023-10-07T23:22:08.197183+0000","last_became_peered":"2023-10-07T23:22:08.197183+0000","last_unstale":"2023-10-07T23:22:09.250603+0000","last_undegraded":"2023-10-07T23:22:09.250603+0000","last_fullsized":"2023-10-07T23:22:09.250603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:13.884964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.241089+0000","last_change":"2023-10-07T23:22:09.229006+0000","last_active":"2023-10-07T23:22:09.241089+0000","last_peered":"2023-10-07T23:22:09.241089+0000","last_clean":"2023-10-07T23:22:09.241089+0000","last_became_active":"2023-10-07T23:22:09.228711+0000","last_became_peered":"2023-10-07T23:22:09.228711+0000","last_unstale":"2023-10-07T23:22:09.241089+0000","last_undegraded":"2023-10-07T23:22:09.241089+0000","last_fullsized":"2023-10-07T23:22:09.241089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:59:53.104146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238362+0000","last_change":"2023-10-07T23:22:08.190945+0000","last_active":"2023-10-07T23:22:09.238362+0000","last_peered":"2023-10-07T23:22:09.238362+0000","last_clean":"2023-10-07T23:22:09.238362+0000","last_became_active":"2023-10-07T23:22:08.190614+0000","last_became_peered":"2023-10-07T23:22:08.190614+0000","last_unstale":"2023-10-07T23:22:09.238362+0000","last_undegraded":"2023-10-07T23:22:09.238362+0000","last_fullsized":"2023-10-07T23:22:09.238362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:24:20.764751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.245047+0000","last_change":"2023-10-07T23:22:09.236038+0000","last_active":"2023-10-07T23:22:09.245047+0000","last_peered":"2023-10-07T23:22:09.245047+0000","last_clean":"2023-10-07T23:22:09.245047+0000","last_became_active":"2023-10-07T23:22:09.235746+0000","last_became_peered":"2023-10-07T23:22:09.235746+0000","last_unstale":"2023-10-07T23:22:09.245047+0000","last_undegraded":"2023-10-07T23:22:09.245047+0000","last_fullsized":"2023-10-07T23:22:09.245047+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:15.289331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.214906+0000","last_change":"2023-10-07T23:22:08.197284+0000","last_active":"2023-10-07T23:22:09.214906+0000","last_peered":"2023-10-07T23:22:09.214906+0000","last_clean":"2023-10-07T23:22:09.214906+0000","last_became_active":"2023-10-07T23:22:08.196962+0000","last_became_peered":"2023-10-07T23:22:08.196962+0000","last_unstale":"2023-10-07T23:22:09.214906+0000","last_undegraded":"2023-10-07T23:22:09.214906+0000","last_fullsized":"2023-10-07T23:22:09.214906+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:04:34.498041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.231103+0000","last_change":"2023-10-07T23:22:09.225903+0000","last_active":"2023-10-07T23:22:09.231103+0000","last_peered":"2023-10-07T23:22:09.231103+0000","last_clean":"2023-10-07T23:22:09.231103+0000","last_became_active":"2023-10-07T23:22:09.225462+0000","last_became_peered":"2023-10-07T23:22:09.225462+0000","last_unstale":"2023-10-07T23:22:09.231103+0000","last_undegraded":"2023-10-07T23:22:09.231103+0000","last_fullsized":"2023-10-07T23:22:09.231103+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:59.534195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217595+0000","last_change":"2023-10-07T23:22:08.199464+0000","last_active":"2023-10-07T23:22:09.217595+0000","last_peered":"2023-10-07T23:22:09.217595+0000","last_clean":"2023-10-07T23:22:09.217595+0000","last_became_active":"2023-10-07T23:22:08.199076+0000","last_became_peered":"2023-10-07T23:22:08.199076+0000","last_unstale":"2023-10-07T23:22:09.217595+0000","last_undegraded":"2023-10-07T23:22:09.217595+0000","last_fullsized":"2023-10-07T23:22:09.217595+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:43:17.251607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.211795+0000","last_change":"2023-10-07T23:22:09.210427+0000","last_active":"2023-10-07T23:22:09.211795+0000","last_peered":"2023-10-07T23:22:09.211795+0000","last_clean":"2023-10-07T23:22:09.211795+0000","last_became_active":"2023-10-07T23:22:09.210044+0000","last_became_peered":"2023-10-07T23:22:09.210044+0000","last_unstale":"2023-10-07T23:22:09.211795+0000","last_undegraded":"2023-10-07T23:22:09.211795+0000","last_fullsized":"2023-10-07T23:22:09.211795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248449+0000","last_change":"2023-10-07T23:22:08.201034+0000","last_active":"2023-10-07T23:22:09.248449+0000","last_peered":"2023-10-07T23:22:09.248449+0000","last_clean":"2023-10-07T23:22:09.248449+0000","last_became_active":"2023-10-07T23:22:08.200733+0000","last_became_peered":"2023-10-07T23:22:08.200733+0000","last_unstale":"2023-10-07T23:22:09.248449+0000","last_undegraded":"2023-10-07T23:22:09.248449+0000","last_fullsized":"2023-10-07T23:22:09.248449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:19.750302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236063+0000","last_change":"2023-10-07T23:22:09.231189+0000","last_active":"2023-10-07T23:22:09.236063+0000","last_peered":"2023-10-07T23:22:09.236063+0000","last_clean":"2023-10-07T23:22:09.236063+0000","last_became_active":"2023-10-07T23:22:09.230905+0000","last_became_peered":"2023-10-07T23:22:09.230905+0000","last_unstale":"2023-10-07T23:22:09.236063+0000","last_undegraded":"2023-10-07T23:22:09.236063+0000","last_fullsized":"2023-10-07T23:22:09.236063+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.247875+0000","last_change":"2023-10-07T23:22:08.199006+0000","last_active":"2023-10-07T23:22:09.247875+0000","last_peered":"2023-10-07T23:22:09.247875+0000","last_clean":"2023-10-07T23:22:09.247875+0000","last_became_active":"2023-10-07T23:22:08.198687+0000","last_became_peered":"2023-10-07T23:22:08.198687+0000","last_unstale":"2023-10-07T23:22:09.247875+0000","last_undegraded":"2023-10-07T23:22:09.247875+0000","last_fullsized":"2023-10-07T23:22:09.247875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:05.995895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232256+0000","last_change":"2023-10-07T23:22:09.227299+0000","last_active":"2023-10-07T23:22:09.232256+0000","last_peered":"2023-10-07T23:22:09.232256+0000","last_clean":"2023-10-07T23:22:09.232256+0000","last_became_active":"2023-10-07T23:22:09.227018+0000","last_became_peered":"2023-10-07T23:22:09.227018+0000","last_unstale":"2023-10-07T23:22:09.232256+0000","last_undegraded":"2023-10-07T23:22:09.232256+0000","last_fullsized":"2023-10-07T23:22:09.232256+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:43.237723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238398+0000","last_change":"2023-10-07T23:22:08.190155+0000","last_active":"2023-10-07T23:22:09.238398+0000","last_peered":"2023-10-07T23:22:09.238398+0000","last_clean":"2023-10-07T23:22:09.238398+0000","last_became_active":"2023-10-07T23:22:08.189890+0000","last_became_peered":"2023-10-07T23:22:08.189890+0000","last_unstale":"2023-10-07T23:22:09.238398+0000","last_undegraded":"2023-10-07T23:22:09.238398+0000","last_fullsized":"2023-10-07T23:22:09.238398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:37.585931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243922+0000","last_change":"2023-10-07T23:22:09.233095+0000","last_active":"2023-10-07T23:22:09.243922+0000","last_peered":"2023-10-07T23:22:09.243922+0000","last_clean":"2023-10-07T23:22:09.243922+0000","last_became_active":"2023-10-07T23:22:09.232792+0000","last_became_peered":"2023-10-07T23:22:09.232792+0000","last_unstale":"2023-10-07T23:22:09.243922+0000","last_undegraded":"2023-10-07T23:22:09.243922+0000","last_fullsized":"2023-10-07T23:22:09.243922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:06.539033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239500+0000","last_change":"2023-10-07T23:22:08.196526+0000","last_active":"2023-10-07T23:22:09.239500+0000","last_peered":"2023-10-07T23:22:09.239500+0000","last_clean":"2023-10-07T23:22:09.239500+0000","last_became_active":"2023-10-07T23:22:08.196260+0000","last_became_peered":"2023-10-07T23:22:08.196260+0000","last_unstale":"2023-10-07T23:22:09.239500+0000","last_undegraded":"2023-10-07T23:22:09.239500+0000","last_fullsized":"2023-10-07T23:22:09.239500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.232300+0000","last_change":"2023-10-07T23:22:09.230763+0000","last_active":"2023-10-07T23:22:09.232300+0000","last_peered":"2023-10-07T23:22:09.232300+0000","last_clean":"2023-10-07T23:22:09.232300+0000","last_became_active":"2023-10-07T23:22:09.230487+0000","last_became_peered":"2023-10-07T23:22:09.230487+0000","last_unstale":"2023-10-07T23:22:09.232300+0000","last_undegraded":"2023-10-07T23:22:09.232300+0000","last_fullsized":"2023-10-07T23:22:09.232300+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:48.735570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.204980+0000","last_change":"2023-10-07T23:22:08.195897+0000","last_active":"2023-10-07T23:22:09.204980+0000","last_peered":"2023-10-07T23:22:09.204980+0000","last_clean":"2023-10-07T23:22:09.204980+0000","last_became_active":"2023-10-07T23:22:08.193256+0000","last_became_peered":"2023-10-07T23:22:08.193256+0000","last_unstale":"2023-10-07T23:22:09.204980+0000","last_undegraded":"2023-10-07T23:22:09.204980+0000","last_fullsized":"2023-10-07T23:22:09.204980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:39.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.225132+0000","last_change":"2023-10-07T23:22:09.222015+0000","last_active":"2023-10-07T23:22:09.225132+0000","last_peered":"2023-10-07T23:22:09.225132+0000","last_clean":"2023-10-07T23:22:09.225132+0000","last_became_active":"2023-10-07T23:22:09.221557+0000","last_became_peered":"2023-10-07T23:22:09.221557+0000","last_unstale":"2023-10-07T23:22:09.225132+0000","last_undegraded":"2023-10-07T23:22:09.225132+0000","last_fullsized":"2023-10-07T23:22:09.225132+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:15.766802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243210+0000","last_change":"2023-10-07T23:22:08.200498+0000","last_active":"2023-10-07T23:22:09.243210+0000","last_peered":"2023-10-07T23:22:09.243210+0000","last_clean":"2023-10-07T23:22:09.243210+0000","last_became_active":"2023-10-07T23:22:08.200099+0000","last_became_peered":"2023-10-07T23:22:08.200099+0000","last_unstale":"2023-10-07T23:22:09.243210+0000","last_undegraded":"2023-10-07T23:22:09.243210+0000","last_fullsized":"2023-10-07T23:22:09.243210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:03.092749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238259+0000","last_change":"2023-10-07T23:22:09.232479+0000","last_active":"2023-10-07T23:22:09.238259+0000","last_peered":"2023-10-07T23:22:09.238259+0000","last_clean":"2023-10-07T23:22:09.238259+0000","last_became_active":"2023-10-07T23:22:09.232186+0000","last_became_peered":"2023-10-07T23:22:09.232186+0000","last_unstale":"2023-10-07T23:22:09.238259+0000","last_undegraded":"2023-10-07T23:22:09.238259+0000","last_fullsized":"2023-10-07T23:22:09.238259+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:56.442036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.208549+0000","last_change":"2023-10-07T23:22:08.200014+0000","last_active":"2023-10-07T23:22:09.208549+0000","last_peered":"2023-10-07T23:22:09.208549+0000","last_clean":"2023-10-07T23:22:09.208549+0000","last_became_active":"2023-10-07T23:22:08.199729+0000","last_became_peered":"2023-10-07T23:22:08.199729+0000","last_unstale":"2023-10-07T23:22:09.208549+0000","last_undegraded":"2023-10-07T23:22:09.208549+0000","last_fullsized":"2023-10-07T23:22:09.208549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.223392+0000","last_change":"2023-10-07T23:22:09.221769+0000","last_active":"2023-10-07T23:22:09.223392+0000","last_peered":"2023-10-07T23:22:09.223392+0000","last_clean":"2023-10-07T23:22:09.223392+0000","last_became_active":"2023-10-07T23:22:09.221362+0000","last_became_peered":"2023-10-07T23:22:09.221362+0000","last_unstale":"2023-10-07T23:22:09.223392+0000","last_undegraded":"2023-10-07T23:22:09.223392+0000","last_fullsized":"2023-10-07T23:22:09.223392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:40.217782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.205982+0000","last_change":"2023-10-07T23:22:08.202898+0000","last_active":"2023-10-07T23:22:09.205982+0000","last_peered":"2023-10-07T23:22:09.205982+0000","last_clean":"2023-10-07T23:22:09.205982+0000","last_became_active":"2023-10-07T23:22:08.202581+0000","last_became_peered":"2023-10-07T23:22:08.202581+0000","last_unstale":"2023-10-07T23:22:09.205982+0000","last_undegraded":"2023-10-07T23:22:09.205982+0000","last_fullsized":"2023-10-07T23:22:09.205982+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:08:14.753423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.235805+0000","last_change":"2023-10-07T23:22:09.233869+0000","last_active":"2023-10-07T23:22:09.235805+0000","last_peered":"2023-10-07T23:22:09.235805+0000","last_clean":"2023-10-07T23:22:09.235805+0000","last_became_active":"2023-10-07T23:22:09.230652+0000","last_became_peered":"2023-10-07T23:22:09.230652+0000","last_unstale":"2023-10-07T23:22:09.235805+0000","last_undegraded":"2023-10-07T23:22:09.235805+0000","last_fullsized":"2023-10-07T23:22:09.235805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:43.237723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.218680+0000","last_change":"2023-10-07T23:22:08.199889+0000","last_active":"2023-10-07T23:22:09.218680+0000","last_peered":"2023-10-07T23:22:09.218680+0000","last_clean":"2023-10-07T23:22:09.218680+0000","last_became_active":"2023-10-07T23:22:08.199599+0000","last_became_peered":"2023-10-07T23:22:08.199599+0000","last_unstale":"2023-10-07T23:22:09.218680+0000","last_undegraded":"2023-10-07T23:22:09.218680+0000","last_fullsized":"2023-10-07T23:22:09.218680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:32:03.635405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.238106+0000","last_change":"2023-10-07T23:22:09.230999+0000","last_active":"2023-10-07T23:22:09.238106+0000","last_peered":"2023-10-07T23:22:09.238106+0000","last_clean":"2023-10-07T23:22:09.238106+0000","last_became_active":"2023-10-07T23:22:09.229998+0000","last_became_peered":"2023-10-07T23:22:09.229998+0000","last_unstale":"2023-10-07T23:22:09.238106+0000","last_undegraded":"2023-10-07T23:22:09.238106+0000","last_fullsized":"2023-10-07T23:22:09.238106+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:25.650167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.243922+0000","last_change":"2023-10-07T23:22:08.198108+0000","last_active":"2023-10-07T23:22:09.243922+0000","last_peered":"2023-10-07T23:22:09.243922+0000","last_clean":"2023-10-07T23:22:09.243922+0000","last_became_active":"2023-10-07T23:22:08.197784+0000","last_became_peered":"2023-10-07T23:22:08.197784+0000","last_unstale":"2023-10-07T23:22:09.243922+0000","last_undegraded":"2023-10-07T23:22:09.243922+0000","last_fullsized":"2023-10-07T23:22:09.243922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:27.339348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248455+0000","last_change":"2023-10-07T23:22:09.243462+0000","last_active":"2023-10-07T23:22:09.248455+0000","last_peered":"2023-10-07T23:22:09.248455+0000","last_clean":"2023-10-07T23:22:09.248455+0000","last_became_active":"2023-10-07T23:22:09.243027+0000","last_became_peered":"2023-10-07T23:22:09.243027+0000","last_unstale":"2023-10-07T23:22:09.248455+0000","last_undegraded":"2023-10-07T23:22:09.248455+0000","last_fullsized":"2023-10-07T23:22:09.248455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:40:02.590917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-07T23:22:10.330210+0000","last_change":"2023-10-07T23:22:08.200713+0000","last_active":"2023-10-07T23:22:10.330210+0000","last_peered":"2023-10-07T23:22:10.330210+0000","last_clean":"2023-10-07T23:22:10.330210+0000","last_became_active":"2023-10-07T23:22:08.200417+0000","last_became_peered":"2023-10-07T23:22:08.200417+0000","last_unstale":"2023-10-07T23:22:10.330210+0000","last_undegraded":"2023-10-07T23:22:10.330210+0000","last_fullsized":"2023-10-07T23:22:10.330210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:47.722190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.239890+0000","last_change":"2023-10-07T23:22:09.237476+0000","last_active":"2023-10-07T23:22:09.239890+0000","last_peered":"2023-10-07T23:22:09.239890+0000","last_clean":"2023-10-07T23:22:09.239890+0000","last_became_active":"2023-10-07T23:22:09.237143+0000","last_became_peered":"2023-10-07T23:22:09.237143+0000","last_unstale":"2023-10-07T23:22:09.239890+0000","last_undegraded":"2023-10-07T23:22:09.239890+0000","last_fullsized":"2023-10-07T23:22:09.239890+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:06.199594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.223505+0000","last_change":"2023-10-07T23:22:08.198547+0000","last_active":"2023-10-07T23:22:09.223505+0000","last_peered":"2023-10-07T23:22:09.223505+0000","last_clean":"2023-10-07T23:22:09.223505+0000","last_became_active":"2023-10-07T23:22:08.197247+0000","last_became_peered":"2023-10-07T23:22:08.197247+0000","last_unstale":"2023-10-07T23:22:09.223505+0000","last_undegraded":"2023-10-07T23:22:09.223505+0000","last_fullsized":"2023-10-07T23:22:09.223505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:10.544875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.212509+0000","last_change":"2023-10-07T23:22:09.211298+0000","last_active":"2023-10-07T23:22:09.212509+0000","last_peered":"2023-10-07T23:22:09.212509+0000","last_clean":"2023-10-07T23:22:09.212509+0000","last_became_active":"2023-10-07T23:22:09.210873+0000","last_became_peered":"2023-10-07T23:22:09.210873+0000","last_unstale":"2023-10-07T23:22:09.212509+0000","last_undegraded":"2023-10-07T23:22:09.212509+0000","last_fullsized":"2023-10-07T23:22:09.212509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:36:10.463096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.203815+0000","last_change":"2023-10-07T23:22:08.196476+0000","last_active":"2023-10-07T23:22:09.203815+0000","last_peered":"2023-10-07T23:22:09.203815+0000","last_clean":"2023-10-07T23:22:09.203815+0000","last_became_active":"2023-10-07T23:22:08.196110+0000","last_became_peered":"2023-10-07T23:22:08.196110+0000","last_unstale":"2023-10-07T23:22:09.203815+0000","last_undegraded":"2023-10-07T23:22:09.203815+0000","last_fullsized":"2023-10-07T23:22:09.203815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:55.428657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.220542+0000","last_change":"2023-10-07T23:22:09.211891+0000","last_active":"2023-10-07T23:22:09.220542+0000","last_peered":"2023-10-07T23:22:09.220542+0000","last_clean":"2023-10-07T23:22:09.220542+0000","last_became_active":"2023-10-07T23:22:09.211518+0000","last_became_peered":"2023-10-07T23:22:09.211518+0000","last_unstale":"2023-10-07T23:22:09.220542+0000","last_undegraded":"2023-10-07T23:22:09.220542+0000","last_fullsized":"2023-10-07T23:22:09.220542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:41.313787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.222745+0000","last_change":"2023-10-07T23:22:08.198487+0000","last_active":"2023-10-07T23:22:09.222745+0000","last_peered":"2023-10-07T23:22:09.222745+0000","last_clean":"2023-10-07T23:22:09.222745+0000","last_became_active":"2023-10-07T23:22:08.198134+0000","last_became_peered":"2023-10-07T23:22:08.198134+0000","last_unstale":"2023-10-07T23:22:09.222745+0000","last_undegraded":"2023-10-07T23:22:09.222745+0000","last_fullsized":"2023-10-07T23:22:09.222745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:04:57.239785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.248570+0000","last_change":"2023-10-07T23:22:09.243348+0000","last_active":"2023-10-07T23:22:09.248570+0000","last_peered":"2023-10-07T23:22:09.248570+0000","last_clean":"2023-10-07T23:22:09.248570+0000","last_became_active":"2023-10-07T23:22:09.240791+0000","last_became_peered":"2023-10-07T23:22:09.240791+0000","last_unstale":"2023-10-07T23:22:09.248570+0000","last_undegraded":"2023-10-07T23:22:09.248570+0000","last_fullsized":"2023-10-07T23:22:09.248570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.206124+0000","last_change":"2023-10-07T23:22:08.192121+0000","last_active":"2023-10-07T23:22:09.206124+0000","last_peered":"2023-10-07T23:22:09.206124+0000","last_clean":"2023-10-07T23:22:09.206124+0000","last_became_active":"2023-10-07T23:22:08.191858+0000","last_became_peered":"2023-10-07T23:22:08.191858+0000","last_unstale":"2023-10-07T23:22:09.206124+0000","last_undegraded":"2023-10-07T23:22:09.206124+0000","last_fullsized":"2023-10-07T23:22:09.206124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:01:39.776359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.237366+0000","last_change":"2023-10-07T23:22:09.232607+0000","last_active":"2023-10-07T23:22:09.237366+0000","last_peered":"2023-10-07T23:22:09.237366+0000","last_clean":"2023-10-07T23:22:09.237366+0000","last_became_active":"2023-10-07T23:22:09.232244+0000","last_became_peered":"2023-10-07T23:22:09.232244+0000","last_unstale":"2023-10-07T23:22:09.237366+0000","last_undegraded":"2023-10-07T23:22:09.237366+0000","last_fullsized":"2023-10-07T23:22:09.237366+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:17:14.898343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.215549+0000","last_change":"2023-10-07T23:22:08.200328+0000","last_active":"2023-10-07T23:22:09.215549+0000","last_peered":"2023-10-07T23:22:09.215549+0000","last_clean":"2023-10-07T23:22:09.215549+0000","last_became_active":"2023-10-07T23:22:08.199980+0000","last_became_peered":"2023-10-07T23:22:08.199980+0000","last_unstale":"2023-10-07T23:22:09.215549+0000","last_undegraded":"2023-10-07T23:22:09.215549+0000","last_fullsized":"2023-10-07T23:22:09.215549+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:40.300407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.236833+0000","last_change":"2023-10-07T23:22:09.234281+0000","last_active":"2023-10-07T23:22:09.236833+0000","last_peered":"2023-10-07T23:22:09.236833+0000","last_clean":"2023-10-07T23:22:09.236833+0000","last_became_active":"2023-10-07T23:22:09.230674+0000","last_became_peered":"2023-10-07T23:22:09.230674+0000","last_unstale":"2023-10-07T23:22:09.236833+0000","last_undegraded":"2023-10-07T23:22:09.236833+0000","last_fullsized":"2023-10-07T23:22:09.236833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:08.166153+0000","last_clean_scrub_stamp":"2023-10-07T23:22:08.166153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:51.824542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.208496+0000","last_change":"2023-10-07T23:22:08.193192+0000","last_active":"2023-10-07T23:22:09.208496+0000","last_peered":"2023-10-07T23:22:09.208496+0000","last_clean":"2023-10-07T23:22:09.208496+0000","last_became_active":"2023-10-07T23:22:08.192927+0000","last_became_peered":"2023-10-07T23:22:08.192927+0000","last_unstale":"2023-10-07T23:22:09.208496+0000","last_undegraded":"2023-10-07T23:22:09.208496+0000","last_fullsized":"2023-10-07T23:22:09.208496+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:22:07.152774+0000","last_clean_scrub_stamp":"2023-10-07T23:22:07.152774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:39.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-07T23:22:09.217208+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:22:09.217208+0000","last_peered":"2023-10-07T23:22:09.217208+0000","last_clean":"2023-10-07T23:22:09.217208+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:22:09.217208+0000","last_undegraded":"2023-10-07T23:22:09.217208+0000","last_fullsized":"2023-10-07T23:22:09.217208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298252,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073588,"statfs":{"total":96636764160,"available":96331354112,"internally_reserved":0,"allocated":987136,"data_stored":739452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]}]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297676,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074164,"statfs":{"total":96636764160,"available":96331943936,"internally_reserved":0,"allocated":397312,"data_stored":149098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":405504,"data_stored":150552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301768,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070072,"statfs":{"total":96636764160,"available":96327753728,"internally_reserved":0,"allocated":393216,"data_stored":149064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.93999999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.755}]},{"osd":3,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.0669999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.153}]},{"osd":6,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.095},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.1140000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:22:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.1950000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302344,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069496,"statfs":{"total":96636764160,"available":96327163904,"internally_reserved":0,"allocated":983040,"data_stored":739362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301780,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070060,"statfs":{"total":96636764160,"available":96327741440,"internally_reserved":0,"allocated":405504,"data_stored":150552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:22:13.571 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T23:22:13.571 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-07T23:22:13.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 Processor -- start 2023-10-07T23:22:13.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 -- start start 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f833815d040 0x7f833815d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833815cbb0 con 0x7f83380af250 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833815a390 con 0x7f833815d040 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8337fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8338159900 con 0x7f83380aee80 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f832e7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f83380af250 0x7f83381591f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36256/0 (socket says 172.21.15.57:36256) 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f832e7fc700 1 -- 172.21.15.57:0/3711757678 learned_addr learned my addr 172.21.15.57:0/3711757678 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f83377fe700 1 --2- 172.21.15.57:0/3711757678 >> v2:172.21.15.154:3300/0 conn(0x7f833815d040 0x7f833815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367651470 0 0) 0x7f833815cbb0 con 0x7f83380af250 2023-10-07T23:22:13.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f8336ffd700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.689+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8324003570 con 0x7f83380af250 2023-10-07T23:22:13.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521218617 0 0) 0x7f8324003570 con 0x7f83380af250 2023-10-07T23:22:13.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 msgr2=0x7f83380c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:13.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 >> v2:172.21.15.154:3300/0 conn(0x7f833815d040 msgr2=0x7f833815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 --2- 172.21.15.57:0/3711757678 >> v2:172.21.15.154:3300/0 conn(0x7f833815d040 0x7f833815d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8338159af0 con 0x7f83380af250 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f8320001470 con 0x7f83380af250 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 >> v1:172.21.15.57:6789/0 conn(0x7f83380af250 legacy=0x7f83381591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8324003a60 0x7f8324003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f8324004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324004d50 0x7f83240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833815a390 con 0x7f8324003a60 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832400ba60 con 0x7f8324004d50 2023-10-07T23:22:13.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832400bba0 con 0x7f83380aee80 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8336ffd700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8324003a60 0x7f8324003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f8324004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83377fe700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324004d50 0x7f83240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 -- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324004d50 msgr2=0x7f83240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8324004d50 0x7f83240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 -- 172.21.15.57:0/3711757678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8324003a60 msgr2=0x7f8324003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8324003a60 0x7f8324003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 -- 172.21.15.57:0/3711757678 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324003570 con 0x7f83380aee80 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8336ffd700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8324003a60 0x7f8324003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f832e7fc700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f8324004810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f832000c3c0 tx=0x7f8320001e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8915edf9dee47 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000cc30 con 0x7f83380aee80 2023-10-07T23:22:13.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000cdb0 con 0x7f83380aee80 2023-10-07T23:22:13.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83367fc700 1 -- 172.21.15.57:0/3711757678 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320005b70 con 0x7f83380aee80 2023-10-07T23:22:13.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 -- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 msgr2=0x7f8324004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 --2- 172.21.15.57:0/3711757678 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f8324004810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f832000c3c0 tx=0x7f8320001e90 comp rx=0 tx=0).stop 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f83377fe700 1 -- 172.21.15.57:0/3711757678 reap_dead start 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 -- 172.21.15.57:0/3711757678 shutdown_connections 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 -- 172.21.15.57:0/3711757678 >> 172.21.15.57:0/3711757678 conn(0x7f833806b5d0 msgr2=0x7f833806b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 -- 172.21.15.57:0/3711757678 shutdown_connections 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.693+0000 7f8337fff700 1 -- 172.21.15.57:0/3711757678 wait complete. 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 Processor -- start 2023-10-07T23:22:13.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 -- start start 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8338156d60 0x7f83381445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338144b20 con 0x7f83381584e0 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338158050 con 0x7f8338156d60 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8337fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83381571d0 con 0x7f83380aee80 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f83377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f83377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56834/0 (socket says 172.21.15.57:56834) 2023-10-07T23:22:13.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f83377fe700 1 -- 172.21.15.57:0/1848980333 learned_addr learned my addr 172.21.15.57:0/1848980333 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:13.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f832e7fc700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8338156d60 0x7f83381445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.697+0000 7f8336ffd700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 -- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 msgr2=0x7f83380af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 -- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8338156d60 msgr2=0x7f83381445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8338156d60 0x7f83381445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 -- 172.21.15.57:0/1848980333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8338157440 con 0x7f83381584e0 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f83377fe700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83300094d0 tx=0x7f833000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000e040 con 0x7f83381584e0 2023-10-07T23:22:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833000f040 con 0x7f83381584e0 2023-10-07T23:22:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8338157720 con 0x7f83381584e0 2023-10-07T23:22:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000fe50 con 0x7f83381584e0 2023-10-07T23:22:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338157ce0 con 0x7f83381584e0 2023-10-07T23:22:13.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.701+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83380711c0 con 0x7f83381584e0 2023-10-07T23:22:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.705+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8330015640 con 0x7f83381584e0 2023-10-07T23:22:13.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.705+0000 7f832f7fe700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 0x7f8318062ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.705+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f833007d000 con 0x7f83381584e0 2023-10-07T23:22:13.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.705+0000 7f8336ffd700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 0x7f8318062ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:13.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.705+0000 7f8336ffd700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 0x7f8318062ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83380c3450 tx=0x7f83280091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:13.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.713+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f833004d730 con 0x7f83381584e0 2023-10-07T23:22:13.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.933+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8338159dd0 con 0x7f83381584e0 2023-10-07T23:22:13.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f832f7fe700 1 -- 172.21.15.57:0/1848980333 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8330050d50 con 0x7f83381584e0 2023-10-07T23:22:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 msgr2=0x7f8318062ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 0x7f8318062ad0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f83380c3450 tx=0x7f83280091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 msgr2=0x7f83380b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:13.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83300094d0 tx=0x7f833000b040 comp rx=0 tx=0).stop 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 shutdown_connections 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f83381584e0 0x7f83380b0380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8318060620 0x7f8318062ad0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8338156d60 0x7f83381445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 --2- 172.21.15.57:0/1848980333 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f83380aee80 0x7f83380af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 >> 172.21.15.57:0/1848980333 conn(0x7f833806b5d0 msgr2=0x7f8338068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 shutdown_connections 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:13.945+0000 7f8337fff700 1 -- 172.21.15.57:0/1848980333 wait complete. 2023-10-07T23:22:13.954 INFO:teuthology.orchestra.run.smithi057.stderr:updated caps for client.0 2023-10-07T23:22:13.968 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:22:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 Processor -- start 2023-10-07T23:22:14.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 -- start start 2023-10-07T23:22:14.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f77e40aefb0 0x7f77e406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 0x7f77e415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e415ccb0 con 0x7f77e40af380 2023-10-07T23:22:14.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e406d1c0 con 0x7f77e40aefb0 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77ea820700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e415cf70 con 0x7f77e406cd40 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e37fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f77e40af380 0x7f77e41599e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36272/0 (socket says 172.21.15.57:36272) 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e37fe700 1 -- 172.21.15.57:0/3889170646 learned_addr learned my addr 172.21.15.57:0/3889170646 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3889170646 >> v2:172.21.15.57:3300/0 conn(0x7f77e40aefb0 0x7f77e406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e8dbd700 1 --2- 172.21.15.57:0/3889170646 >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 0x7f77e415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942934782 0 0) 0x7f77e415ccb0 con 0x7f77e40af380 2023-10-07T23:22:14.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d0003570 con 0x7f77e40af380 2023-10-07T23:22:14.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.069+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3948983414 0 0) 0x7f77d0003570 con 0x7f77e40af380 2023-10-07T23:22:14.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e3fff700 1 -- 172.21.15.57:0/3889170646 >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 msgr2=0x7f77e415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3889170646 >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 0x7f77e415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e3fff700 1 -- 172.21.15.57:0/3889170646 >> v1:172.21.15.57:6789/0 conn(0x7f77e40af380 legacy=0x7f77e41599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e3fff700 1 -- 172.21.15.57:0/3889170646 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e415d250 con 0x7f77e40aefb0 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e8dbd700 1 --2- 172.21.15.57:0/3889170646 >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 0x7f77e415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3889170646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e40aefb0 0x7f77e406c800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f77e40af820 tx=0x7f77cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919121b55835ffd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4001470 con 0x7f77e40aefb0 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc011420 con 0x7f77e40aefb0 2023-10-07T23:22:14.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77e2ffd700 1 -- 172.21.15.57:0/3889170646 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0117a0 con 0x7f77e40aefb0 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 -- 172.21.15.57:0/3889170646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e40aefb0 msgr2=0x7f77e406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 --2- 172.21.15.57:0/3889170646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e40aefb0 0x7f77e406c800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f77e40af820 tx=0x7f77cc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 -- 172.21.15.57:0/3889170646 shutdown_connections 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 --2- 172.21.15.57:0/3889170646 >> v2:172.21.15.154:3300/0 conn(0x7f77e406cd40 0x7f77e415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 --2- 172.21.15.57:0/3889170646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e40aefb0 0x7f77e406c800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 -- 172.21.15.57:0/3889170646 >> 172.21.15.57:0/3889170646 conn(0x7f77e4068630 msgr2=0x7f77e4068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 -- 172.21.15.57:0/3889170646 shutdown_connections 2023-10-07T23:22:14.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.073+0000 7f77ea820700 1 -- 172.21.15.57:0/3889170646 wait complete. 2023-10-07T23:22:14.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 Processor -- start 2023-10-07T23:22:14.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- start start 2023-10-07T23:22:14.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e40aefb0 0x7f77e415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e40af380 0x7f77e4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4154410 con 0x7f77e406cd40 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e41546b0 con 0x7f77e40af380 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4154990 con 0x7f77e40aefb0 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56856/0 (socket says 172.21.15.57:56856) 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 -- 172.21.15.57:0/3716514603 learned_addr learned my addr 172.21.15.57:0/3716514603 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e8dbd700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e40af380 0x7f77e4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e37fe700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e40aefb0 0x7f77e415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 -- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e40aefb0 msgr2=0x7f77e415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e40aefb0 0x7f77e415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 -- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e40af380 msgr2=0x7f77e4158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e40af380 0x7f77e4158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 -- 172.21.15.57:0/3716514603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc009a20 con 0x7f77e406cd40 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77e3fff700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77e40c6f70 tx=0x7f77cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01cc30 con 0x7f77e406cd40 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc01cd90 con 0x7f77e406cd40 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc009420 con 0x7f77e406cd40 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01a6a0 con 0x7f77e406cd40 2023-10-07T23:22:14.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.077+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4154f50 con 0x7f77e406cd40 2023-10-07T23:22:14.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e40711c0 con 0x7f77e406cd40 2023-10-07T23:22:14.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f77cc01aa50 con 0x7f77e406cd40 2023-10-07T23:22:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77dbfff700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 0x7f77c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f77cc07e130 con 0x7f77e406cd40 2023-10-07T23:22:14.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77e37fe700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 0x7f77c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.081+0000 7f77e37fe700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 0x7f77c4062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77d4001620 tx=0x7f77d40014c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.089+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f77cc04e720 con 0x7f77e406cd40 2023-10-07T23:22:14.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.285+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e40cd760 con 0x7f77e406cd40 2023-10-07T23:22:14.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.285+0000 7f77dbfff700 1 -- 172.21.15.57:0/3716514603 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f77cc051d40 con 0x7f77e406cd40 2023-10-07T23:22:14.293 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:22:14.293 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:22:14.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 msgr2=0x7f77c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 0x7f77c4062a00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77d4001620 tx=0x7f77d40014c0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 msgr2=0x7f77e4152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77e40c6f70 tx=0x7f77cc016000 comp rx=0 tx=0).stop 2023-10-07T23:22:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 shutdown_connections 2023-10-07T23:22:14.296 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f77c4060550 0x7f77c4062a00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77e40af380 0x7f77e4158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f77e40aefb0 0x7f77e415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 --2- 172.21.15.57:0/3716514603 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f77e406cd40 0x7f77e4152f40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 >> 172.21.15.57:0/3716514603 conn(0x7f77e4068630 msgr2=0x7f77e40cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.297 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 shutdown_connections 2023-10-07T23:22:14.298 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.289+0000 7f77ea820700 1 -- 172.21.15.57:0/3716514603 wait complete. 2023-10-07T23:22:14.298 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:22:14.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T23:22:08.542997+0000', 'modified': '2023-10-07T23:22:12.219532+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4956}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4956': {'gid': 4956, 'name': 'c', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.57:6835/2270357685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6834', 'nonce': 2270357685}, {'type': 'v1', 'addr': '172.21.15.57:6835', 'nonce': 2270357685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T23:22:14.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T23:22:14.710 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-07T23:22:14.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 Processor -- start 2023-10-07T23:22:14.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 -- start start 2023-10-07T23:22:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 0x7f50f40c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50f4159130 0x7f50f40c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f40c83b0 con 0x7f50f4158d60 2023-10-07T23:22:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f40c8690 con 0x7f50f4159130 2023-10-07T23:22:14.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f97df700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f40c8970 con 0x7f50f406bc90 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f50f4158d60 0x7f50f406b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36288/0 (socket says 172.21.15.57:36288) 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f3fff700 1 -- 172.21.15.57:0/2114044545 learned_addr learned my addr 172.21.15.57:0/2114044545 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.825+0000 7f50f37fe700 1 --2- 172.21.15.57:0/2114044545 >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 0x7f50f40c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2114044545 >> v2:172.21.15.57:3300/0 conn(0x7f50f4159130 0x7f50f40c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 -- 172.21.15.57:0/2114044545 >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 msgr2=0x7f50f40c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2114044545 >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 0x7f50f40c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 -- 172.21.15.57:0/2114044545 >> v1:172.21.15.57:6789/0 conn(0x7f50f4158d60 legacy=0x7f50f406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 -- 172.21.15.57:0/2114044545 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f40c8c50 con 0x7f50f4159130 2023-10-07T23:22:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f37fe700 1 --2- 172.21.15.57:0/2114044545 >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 0x7f50f40c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:14.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2114044545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40c7ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50f40ab060 tx=0x7f50ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d22b435ee407b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f2ffd700 1 -- 172.21.15.57:0/2114044545 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec011630 con 0x7f50f4159130 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f2ffd700 1 -- 172.21.15.57:0/2114044545 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec011790 con 0x7f50f4159130 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f2ffd700 1 -- 172.21.15.57:0/2114044545 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec011b30 con 0x7f50f4159130 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- 172.21.15.57:0/2114044545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 msgr2=0x7f50f40c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 --2- 172.21.15.57:0/2114044545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40c7ac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50f40ab060 tx=0x7f50ec0091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- 172.21.15.57:0/2114044545 shutdown_connections 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 --2- 172.21.15.57:0/2114044545 >> v2:172.21.15.154:3300/0 conn(0x7f50f406bc90 0x7f50f40c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 --2- 172.21.15.57:0/2114044545 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40c7ac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- 172.21.15.57:0/2114044545 >> 172.21.15.57:0/2114044545 conn(0x7f50f40670e0 msgr2=0x7f50f40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- 172.21.15.57:0/2114044545 shutdown_connections 2023-10-07T23:22:14.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- 172.21.15.57:0/2114044545 wait complete. 2023-10-07T23:22:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 Processor -- start 2023-10-07T23:22:14.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.829+0000 7f50f97df700 1 -- start start 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50f406bc90 0x7f50f40b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f4158d60 0x7f50f40ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40b1150 con 0x7f50f4159130 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40c2f60 con 0x7f50f4158d60 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40c3240 con 0x7f50f406bc90 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56878/0 (socket says 172.21.15.57:56878) 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f4158d60 0x7f50f40ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 -- 172.21.15.57:0/2154622315 learned_addr learned my addr 172.21.15.57:0/2154622315 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f3fff700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50f406bc90 0x7f50f40b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 -- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50f406bc90 msgr2=0x7f50f40b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50f406bc90 0x7f50f40b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 -- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f4158d60 msgr2=0x7f50f40ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f4158d60 0x7f50f40ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 -- 172.21.15.57:0/2154622315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4009a20 con 0x7f50f4159130 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f8fde700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50f406ec70 tx=0x7f50ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50dbfff700 1 -- 172.21.15.57:0/2154622315 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec011b10 con 0x7f50f4159130 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50dbfff700 1 -- 172.21.15.57:0/2154622315 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec0253f0 con 0x7f50f4159130 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec009420 con 0x7f50f4159130 2023-10-07T23:22:14.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50dbfff700 1 -- 172.21.15.57:0/2154622315 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0257b0 con 0x7f50f4159130 2023-10-07T23:22:14.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.833+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f40c3800 con 0x7f50f4159130 2023-10-07T23:22:14.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50dbfff700 1 -- 172.21.15.57:0/2154622315 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f50ec025ba0 con 0x7f50f4159130 2023-10-07T23:22:14.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 Processor -- start 2023-10-07T23:22:14.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 -- start start 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50dbfff700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50dbfff700 1 -- 172.21.15.57:0/2154622315 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50ec07e190 con 0x7f50f4159130 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x2607440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x23d4790 0x2607980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x2534f30 con 0x2389ae0 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x2530ed0 con 0x23d4790 2023-10-07T23:22:14.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x24f70b0 con 0x2389710 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f37fe700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f37fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x2607440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56884/0 (socket says 172.21.15.57:56884) 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 -- 172.21.15.57:0/867670850 learned_addr learned my addr 172.21.15.57:0/867670850 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f8fde700 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x23d4790 0x2607980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 -- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 msgr2=0x2607440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x2607440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 -- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x23d4790 msgr2=0x2607980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x23d4790 0x2607980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.837+0000 7f50f3fff700 1 -- 172.21.15.57:0/867670850 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x23d59e0 con 0x2389ae0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50f37fe700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009fd0 tx=0x7f50e400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50f3fff700 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50e8004df0 tx=0x7f50e8007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286748fc8834fb3b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50d97fa700 1 -- 172.21.15.57:0/867670850 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8001560 con 0x2389ae0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50d97fa700 1 -- 172.21.15.57:0/867670850 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e80016c0 con 0x2389ae0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50d97fa700 1 -- 172.21.15.57:0/867670850 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e8001a80 con 0x2389ae0 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- 172.21.15.57:0/867670850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 msgr2=0x2606f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50e8004df0 tx=0x7f50e8007eb0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- 172.21.15.57:0/867670850 shutdown_connections 2023-10-07T23:22:14.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x23d4790 0x2607980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x2607440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 --2- 172.21.15.57:0/867670850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389ae0 0x2606f00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- 172.21.15.57:0/867670850 >> 172.21.15.57:0/867670850 conn(0x2389eb0 msgr2=0x25c3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- 172.21.15.57:0/867670850 shutdown_connections 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- 172.21.15.57:0/867670850 wait complete. 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 -- start start 2023-10-07T23:22:14.849 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.841+0000 7f50fccea740 1 Processor -- start 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x260b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x260bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x24f6d50 con 0x23d4790 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x24cf970 con 0x2389ae0 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x24f9420 con 0x2389710 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 10 client.0 _ll_register_callbacks cb 0x257a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.853 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x260bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25eccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56886/0 (socket says 172.21.15.57:56886) 2023-10-07T23:22:14.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x260b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.854 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f8fde700 1 -- 172.21.15.57:0/1947064119 learned_addr learned my addr 172.21.15.57:0/1947064119 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 -- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 msgr2=0x260b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x260b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 -- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 msgr2=0x25eccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25eccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 -- 172.21.15.57:0/1947064119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f40c83b0 con 0x2389ae0 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f3fff700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x260b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50f37fe700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x260bad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50f406c090 tx=0x7f50e400e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1bbfeca93bd1021 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400e070 con 0x2389ae0 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mon.1 v2:172.21.15.154:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50e4021040 con 0x2389ae0 2023-10-07T23:22:14.857 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x24cfcc0 con 0x2389ae0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50e4008b30 con 0x2389ae0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 10 client.5110 fetch_fsmap learned FSMap version 13 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x24ff370 con 0x2389ae0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mon.1 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x24cfcc0 con 0x2389ae0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mon.1 v2:172.21.15.154:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50e4007bf0 con 0x2389ae0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 10 client.5110 fetch_fsmap finished waiting for FSMap version 13 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 10 client.5110 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.845+0000 7f50fccea740 20 client.5110 populate_metadata read hostname 'smithi057' 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25ee2e0 0x26166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 4 client.5110 mds_command: new command op to 4956 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 --> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x24d33e0 con 0x25ee2e0 2023-10-07T23:22:14.858 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50f3fff700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25ee2e0 0x26166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50d27fc700 10 client.5110 ms_handle_connect on v2:172.21.15.57:6834/2270357685 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50f3fff700 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25ee2e0 0x26166b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c85ff101081c9 server_cookie=9c3a5fb007d924ab in_seq=0 out_seq=0 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50d27fc700 1 -- 172.21.15.57:0/1947064119 <== mds.0 v2:172.21.15.57:6834/2270357685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x24d33e0 con 0x25ee2e0 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50d27fc700 10 client.5110 handle_command_reply: tid=0 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 client.5110 shutdown 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 msgr2=0x260bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x260bad0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50f406c090 tx=0x7f50e400e000 comp rx=0 tx=0).stop 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 shutdown_connections 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25ee2e0 0x26166b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x260bad0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 --2- 172.21.15.57:0/1947064119 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x260b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50d27fc700 0 client.5110 ms_handle_reset on v2:172.21.15.57:6834/2270357685 2023-10-07T23:22:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 >> 172.21.15.57:0/1947064119 conn(0x2389eb0 msgr2=0x25bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 shutdown_connections 2023-10-07T23:22:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 1 -- 172.21.15.57:0/1947064119 wait complete. 2023-10-07T23:22:14.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.849+0000 7f50fccea740 20 client.5110 trim_cache size 0 max 16384 2023-10-07T23:22:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 Processor -- start 2023-10-07T23:22:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- start start 2023-10-07T23:22:14.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x25bdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x25be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x2505610 con 0x23d4790 2023-10-07T23:22:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x24027c0 con 0x2389ae0 2023-10-07T23:22:14.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x252e890 con 0x2389710 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56888/0 (socket says 172.21.15.57:56888) 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x25be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x25bdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 -- 172.21.15.57:0/1416748592 learned_addr learned my addr 172.21.15.57:0/1416748592 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.876 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 -- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 msgr2=0x25bdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x25bdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 -- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 msgr2=0x25be010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x25be010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 -- 172.21.15.57:0/1416748592 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x24e5d80 con 0x23d4790 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50f8fde700 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f50ec052330 tx=0x7f50ec055040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a669f35e52366d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50d9ffb700 1 -- 172.21.15.57:0/1416748592 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec052170 con 0x23d4790 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50d9ffb700 1 -- 172.21.15.57:0/1416748592 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec001d20 con 0x23d4790 2023-10-07T23:22:14.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50d9ffb700 1 -- 172.21.15.57:0/1416748592 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec052ae0 con 0x23d4790 2023-10-07T23:22:14.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- 172.21.15.57:0/1416748592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 msgr2=0x25c1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f50ec052330 tx=0x7f50ec055040 comp rx=0 tx=0).stop 2023-10-07T23:22:14.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- 172.21.15.57:0/1416748592 shutdown_connections 2023-10-07T23:22:14.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x23d4790 0x25c1500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x25be010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 --2- 172.21.15.57:0/1416748592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x2389710 0x25bdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- 172.21.15.57:0/1416748592 >> 172.21.15.57:0/1416748592 conn(0x2389eb0 msgr2=0x25be820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- 172.21.15.57:0/1416748592 shutdown_connections 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.869+0000 7f50fccea740 1 -- 172.21.15.57:0/1416748592 wait complete. 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 -- start start 2023-10-07T23:22:14.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 Processor -- start 2023-10-07T23:22:14.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x2660800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 0x25e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x252cb50 con 0x2389710 2023-10-07T23:22:14.880 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x24d33e0 con 0x2389ae0 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x2534100 con 0x23d4790 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50fccea740 10 client.0 _ll_register_callbacks cb 0x23559d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56894/0 (socket says 172.21.15.57:56894) 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f8fde700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 0x25e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 -- 172.21.15.57:0/3900493954 learned_addr learned my addr 172.21.15.57:0/3900493954 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:14.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f37fe700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x2660800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 -- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 msgr2=0x25e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 0x25e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 -- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 msgr2=0x2660800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x2660800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f3fff700 1 -- 172.21.15.57:0/3900493954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f40c3800 con 0x2389710 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.873+0000 7f50f8fde700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 0x25e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f3fff700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50e8001bf0 tx=0x7f50e8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bab6061b1e5dd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:14.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e802e030 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50e802f070 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50e80016c0 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50e8025040 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 10 client.5139 fetch_fsmap learned FSMap version 13 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x24da130 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e80016c0 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f50e8012070 con 0x2389710 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 10 client.5139 fetch_fsmap finished waiting for FSMap version 13 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 10 client.5139 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 20 client.5139 populate_metadata read hostname 'smithi057' 2023-10-07T23:22:14.884 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25e80c0 0x25e8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:14.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 4 client.5139 mds_command: new command op to 4956 tid=0[{"prefix": "status"}] 2023-10-07T23:22:14.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 --> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] -- command(tid 0: {"prefix": "status"}) v1 -- 0x2505410 con 0x25e80c0 2023-10-07T23:22:14.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f37fe700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25e80c0 0x25e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:14.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f37fe700 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25e80c0 0x25e8490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba173c53aed09c7 server_cookie=55242f58a0b15703 in_seq=0 out_seq=0 2023-10-07T23:22:14.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 10 client.5139 ms_handle_connect on v2:172.21.15.57:6834/2270357685 2023-10-07T23:22:14.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 1 -- 172.21.15.57:0/3900493954 <== mds.0 v2:172.21.15.57:6834/2270357685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x2505410 con 0x25e80c0 2023-10-07T23:22:14.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 10 client.5139 handle_command_reply: tid=0 2023-10-07T23:22:14.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 client.5139 shutdown 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 msgr2=0x26602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50e8001bf0 tx=0x7f50e8021040 comp rx=0 tx=0).stop 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 shutdown_connections 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0x25e80c0 0x25e8490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x23d4790 0x25e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x2389ae0 0x2660800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 --2- 172.21.15.57:0/3900493954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x2389710 0x26602c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50d27fc700 0 client.5139 ms_handle_reset on v2:172.21.15.57:6834/2270357685 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 >> 172.21.15.57:0/3900493954 conn(0x2389eb0 msgr2=0x25bc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 shutdown_connections 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 1 -- 172.21.15.57:0/3900493954 wait complete. 2023-10-07T23:22:14.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50fccea740 20 client.5139 trim_cache size 0 max 16384 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "aaf09e93-912b-470e-9df7-bb9a606f50e4", 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 4956, 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 5.6684599110000002, 2023-10-07T23:22:14.888 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 13, 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 35, 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 35, 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 13.581284238 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 msgr2=0x7f50e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009fd0 tx=0x7f50e400b040 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 msgr2=0x7f50f40ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50f406ec70 tx=0x7f50ec017000 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 shutdown_connections 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50f4159130 0x7f50f40ca6c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50e0060550 0x7f50e0062a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f4158d60 0x7f50f40ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 --2- 172.21.15.57:0/2154622315 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50f406bc90 0x7f50f40b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.877+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 >> 172.21.15.57:0/2154622315 conn(0x7f50f40670e0 msgr2=0x7f50f40674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.881+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 shutdown_connections 2023-10-07T23:22:14.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:14.881+0000 7f50f97df700 1 -- 172.21.15.57:0/2154622315 wait complete. 2023-10-07T23:22:14.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:22:15.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 Processor -- start 2023-10-07T23:22:15.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- start start 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f069c0c7800 0x7f069c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c06d750 con 0x7f069c1583f0 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0c8b80 con 0x7f069c0c7800 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c7f70 con 0x7f069c068530 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f069c0c7800 0x7f069c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a16e5700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f069c1583f0 0x7f069c158fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36302/0 (socket says 172.21.15.57:36302) 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a16e5700 1 -- 172.21.15.57:0/2457234259 learned_addr learned my addr 172.21.15.57:0/2457234259 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:15.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a0ee4700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 -- 172.21.15.57:0/2457234259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 msgr2=0x7f069c068900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:15.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c068900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 -- 172.21.15.57:0/2457234259 >> v1:172.21.15.57:6789/0 conn(0x7f069c1583f0 legacy=0x7f069c158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 -- 172.21.15.57:0/2457234259 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0c8250 con 0x7f069c0c7800 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069b7fe700 1 -- 172.21.15.57:0/2457234259 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811566046 0 0) 0x7f0694001110 con 0x7f069c1583f0 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a0ee4700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069bfff700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c06d210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e60 tx=0x7f0690009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f3acdde1e8fa05 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069b7fe700 1 -- 172.21.15.57:0/2457234259 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000e070 con 0x7f069c0c7800 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069b7fe700 1 -- 172.21.15.57:0/2457234259 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000bc30 con 0x7f069c0c7800 2023-10-07T23:22:15.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f069b7fe700 1 -- 172.21.15.57:0/2457234259 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690012430 con 0x7f069c0c7800 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- 172.21.15.57:0/2457234259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 msgr2=0x7f069c06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c06d210 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e60 tx=0x7f0690009780 comp rx=0 tx=0).stop 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- 172.21.15.57:0/2457234259 shutdown_connections 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c06d210 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 --2- 172.21.15.57:0/2457234259 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c068900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- 172.21.15.57:0/2457234259 >> 172.21.15.57:0/2457234259 conn(0x7f069c0cce30 msgr2=0x7f069c0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- 172.21.15.57:0/2457234259 shutdown_connections 2023-10-07T23:22:15.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.009+0000 7f06a3148700 1 -- 172.21.15.57:0/2457234259 wait complete. 2023-10-07T23:22:15.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 Processor -- start 2023-10-07T23:22:15.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 -- start start 2023-10-07T23:22:15.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 0x7f069c06b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c151620 con 0x7f069c0c7800 2023-10-07T23:22:15.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c155f70 con 0x7f069c1583f0 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.013+0000 7f06a3148700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c5cc0 con 0x7f069c068530 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56908/0 (socket says 172.21.15.57:56908) 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 -- 172.21.15.57:0/1436044188 learned_addr learned my addr 172.21.15.57:0/1436044188 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a0ee4700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a16e5700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 0x7f069c06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 -- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 msgr2=0x7f069c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 -- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 msgr2=0x7f069c06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 0x7f069c06b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 -- 172.21.15.57:0/1436044188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009a20 con 0x7f069c0c7800 2023-10-07T23:22:15.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a16e5700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 0x7f069c06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:22:15.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f069bfff700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f06900099f0 tx=0x7f0690019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:15.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690012db0 con 0x7f069c0c7800 2023-10-07T23:22:15.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0690027470 con 0x7f069c0c7800 2023-10-07T23:22:15.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c0c5fa0 con 0x7f069c0c7800 2023-10-07T23:22:15.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690027830 con 0x7f069c0c7800 2023-10-07T23:22:15.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c155c00 con 0x7f069c0c7800 2023-10-07T23:22:15.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.017+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c156100 con 0x7f069c0c7800 2023-10-07T23:22:15.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.021+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0690024070 con 0x7f069c0c7800 2023-10-07T23:22:15.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.021+0000 7f0698ff9700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 0x7f0688062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:15.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.021+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f069007ee40 con 0x7f069c0c7800 2023-10-07T23:22:15.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.021+0000 7f06a0ee4700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 0x7f0688062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:15.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.021+0000 7f06a0ee4700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 0x7f0688062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f068c006f00 tx=0x7f068c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:15.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.029+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f069004f430 con 0x7f069c0c7800 2023-10-07T23:22:15.234 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.229+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f069c0711c0 con 0x7f069c0c7800 2023-10-07T23:22:15.235 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.229+0000 7f0698ff9700 1 -- 172.21.15.57:0/1436044188 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0690052a50 con 0x7f069c0c7800 2023-10-07T23:22:15.235 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:22:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 msgr2=0x7f0688062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 0x7f0688062a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f068c006f00 tx=0x7f068c008040 comp rx=0 tx=0).stop 2023-10-07T23:22:15.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 msgr2=0x7f069c0c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f06900099f0 tx=0x7f0690019000 comp rx=0 tx=0).stop 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 shutdown_connections 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0688060550 0x7f0688062a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f069c1583f0 0x7f069c06b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f069c0c7800 0x7f069c0c4320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 --2- 172.21.15.57:0/1436044188 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f069c068530 0x7f069c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 >> 172.21.15.57:0/1436044188 conn(0x7f069c0cce30 msgr2=0x7f069c153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 shutdown_connections 2023-10-07T23:22:15.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:15.233+0000 7f06a3148700 1 -- 172.21.15.57:0/1436044188 wait complete. 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:22:15.253 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:22:15.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.254 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.259 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42500sec preferred_lft 42500sec 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:22:15.260 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 02:42:22:80:74:34 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::88e7:63ff:fe49:2a21/64 scope link 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:22:15.261 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 5a:76:86:e5:a8:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::5876:86ff:fee5:a805/64 scope link 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::c75:b1ff:fef6:552a/64 scope link 2023-10-07T23:22:15.262 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:15.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.265 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:22:15.315 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-07T23:22:15.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.317 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-07T23:22:15.326 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-07T23:22:15.326 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:22:15.326 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-07T23:22:15.326 INFO:teuthology.orchestra.run.smithi057.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:22:15.327 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-07T23:22:15.327 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:22:15.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.328 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:22:15.329 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:22:15.329 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T23:22:15.329 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T23:22:15.329 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T23:22:15.330 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:22:15.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.372 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:22:15.420 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T23:22:15.420 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:22:15.420 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:22:15.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:15.420 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-07T23:22:15.437 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T23:22:15.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:22:15.437 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:15.445 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:22:15.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:22:15.445 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:15.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:22:15.511 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:15.511 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:15.526 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:15.527 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:15.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:15.528 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:22:15.534 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:22:15.534 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T23:22:15.579 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:22:15.597 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:15.597 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:15.606 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:15.606 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:15.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:15.607 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:22:15.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:22:15.661+0000 7f82a2cbb380 -1 init, newargv = 0x5573457e4270 newargc=15 2023-10-07T23:22:15.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[41926]: starting ceph client 2023-10-07T23:22:15.676 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[41926]: starting fuse 2023-10-07T23:22:16.713 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:22:16.732 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:16.732 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:16.789 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:16.789 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:16.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:16.790 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:22:16.798 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:22:16.798 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:22:16.798 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:16.798 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:22:16.798 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T23:22:16.799 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:22:16.800 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-10-07T23:22:16.801 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:16.886 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.41926.asok 2023-10-07T23:22:17.069 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41926.asok 2023-10-07T23:22:17.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:17.070 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41926.asok status 2023-10-07T23:22:17.213 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:22:17.213 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "41926", 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-07T23:22:17.214 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5157, 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5157 2023-10-07T23:22:17.215 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 831190132 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 831190132 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:17.216 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5157 192.168.0.1:0/831190132", 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/831190132", 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 13, 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 35, 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-07T23:22:17.217 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:22:17.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:17.223 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-07T23:22:17.223 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:17.228 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-07T23:22:17.228 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:17.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:17.229 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:22:17.244 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:22:17.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 Processor -- start 2023-10-07T23:22:17.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 -- start start 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 0x7f57fc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f57fc15d2a0 0x7f57fc0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc06bbf0 con 0x7f57fc15ced0 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc0c9010 con 0x7f57fc15d2a0 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc06b020 con 0x7f57fc161a70 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57faffd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f57fc15ced0 0x7f57fc0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47108/0 (socket says 172.21.15.57:47108) 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57faffd700 1 -- 172.21.15.57:0/1510079555 learned_addr learned my addr 172.21.15.57:0/1510079555 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fb7fe700 1 --2- 172.21.15.57:0/1510079555 >> v2:172.21.15.154:3300/0 conn(0x7f57fc15d2a0 0x7f57fc0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 --2- 172.21.15.57:0/1510079555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 0x7f57fc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57f9ffb700 1 -- 172.21.15.57:0/1510079555 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247943370 0 0) 0x7f57fc06bbf0 con 0x7f57fc15ced0 2023-10-07T23:22:17.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57f9ffb700 1 -- 172.21.15.57:0/1510079555 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e4003570 con 0x7f57fc15ced0 2023-10-07T23:22:17.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1510079555 >> v2:172.21.15.154:3300/0 conn(0x7f57fc15d2a0 msgr2=0x7f57fc0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:17.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 --2- 172.21.15.57:0/1510079555 >> v2:172.21.15.154:3300/0 conn(0x7f57fc15d2a0 0x7f57fc0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1510079555 >> v1:172.21.15.57:6789/0 conn(0x7f57fc15ced0 legacy=0x7f57fc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 -- 172.21.15.57:0/1510079555 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc06b2c0 con 0x7f57fc161a70 2023-10-07T23:22:17.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.345+0000 7f57fa7fc700 1 --2- 172.21.15.57:0/1510079555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 0x7f57fc06c080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d447c78017ec8f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:17.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57f9ffb700 1 -- 172.21.15.57:0/1510079555 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00e070 con 0x7f57fc161a70 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57f9ffb700 1 -- 172.21.15.57:0/1510079555 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00bab0 con 0x7f57fc161a70 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57f9ffb700 1 -- 172.21.15.57:0/1510079555 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00be30 con 0x7f57fc161a70 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- 172.21.15.57:0/1510079555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 msgr2=0x7f57fc06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- 172.21.15.57:0/1510079555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 0x7f57fc06c080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec009780 comp rx=0 tx=0).stop 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- 172.21.15.57:0/1510079555 shutdown_connections 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- 172.21.15.57:0/1510079555 >> v2:172.21.15.154:3300/0 conn(0x7f57fc15d2a0 0x7f57fc0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- 172.21.15.57:0/1510079555 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc161a70 0x7f57fc06c080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- 172.21.15.57:0/1510079555 >> 172.21.15.57:0/1510079555 conn(0x7f57fc0670e0 msgr2=0x7f57fc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- 172.21.15.57:0/1510079555 shutdown_connections 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- 172.21.15.57:0/1510079555 wait complete. 2023-10-07T23:22:17.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 Processor -- start 2023-10-07T23:22:17.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- start start 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc15ced0 0x7f57fc15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57fc15d2a0 0x7f57fc0c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c2420 con 0x7f57fc161a70 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15d7f0 con 0x7f57fc15d2a0 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fbfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15da40 con 0x7f57fc15ced0 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fb7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41364/0 (socket says 172.21.15.57:41364) 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fb7fe700 1 -- 172.21.15.57:0/2544116910 learned_addr learned my addr 172.21.15.57:0/2544116910 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57faffd700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc15ced0 0x7f57fc15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.349+0000 7f57fa7fc700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57fc15d2a0 0x7f57fc0c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 -- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc15ced0 msgr2=0x7f57fc15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc15ced0 0x7f57fc15fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 -- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57fc15d2a0 msgr2=0x7f57fc0c1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57fc15d2a0 0x7f57fc0c1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 -- 172.21.15.57:0/2544116910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a20 con 0x7f57fc161a70 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fb7fe700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f57f400ea50 tx=0x7f57f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400ee90 con 0x7f57fc161a70 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f400c850 con 0x7f57fc161a70 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f400cc10 con 0x7f57fc161a70 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc15dd20 con 0x7f57fc161a70 2023-10-07T23:22:17.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc15e2e0 con 0x7f57fc161a70 2023-10-07T23:22:17.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc0711c0 con 0x7f57fc161a70 2023-10-07T23:22:17.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.353+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f57f4013470 con 0x7f57fc161a70 2023-10-07T23:22:17.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.357+0000 7f57e2ffd700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 0x7f57e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:17.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.357+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57f407ca00 con 0x7f57fc161a70 2023-10-07T23:22:17.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.357+0000 7f57faffd700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 0x7f57e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:17.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.357+0000 7f57faffd700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 0x7f57e8062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57f0007fd0 tx=0x7f57f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:17.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.361+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f57f404cff0 con 0x7f57fc161a70 2023-10-07T23:22:17.564 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.557+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57fc0cc3d0 con 0x7f57fc161a70 2023-10-07T23:22:17.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.557+0000 7f57e2ffd700 1 -- 172.21.15.57:0/2544116910 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f57f4050610 con 0x7f57fc161a70 2023-10-07T23:22:17.565 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:22:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 msgr2=0x7f57e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 0x7f57e8062a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57f0007fd0 tx=0x7f57f0009040 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 msgr2=0x7f57fc0c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f57f400ea50 tx=0x7f57f40091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 shutdown_connections 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f57fc161a70 0x7f57fc0c1770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f57e8060550 0x7f57e8062a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57fc15d2a0 0x7f57fc0c1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 --2- 172.21.15.57:0/2544116910 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f57fc15ced0 0x7f57fc15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 >> 172.21.15.57:0/2544116910 conn(0x7f57fc0670e0 msgr2=0x7f57fc153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 shutdown_connections 2023-10-07T23:22:17.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:17.561+0000 7f57fbfff700 1 -- 172.21.15.57:0/2544116910 wait complete. 2023-10-07T23:22:17.579 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:22:17.580 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:22:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.581 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-07T23:22:17.589 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:22:17.589 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:22:17.589 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.590 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:9b:82:8d:1f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:22:17.591 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::4a5:81ff:fe8f:3c66/64 scope link 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::74ca:eaff:feca:a10e/64 scope link 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether de:fc:19:2e:9b:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T23:22:17.592 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::dcfc:19ff:fe2e:9bcb/64 scope link 2023-10-07T23:22:17.593 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:22:17.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.593 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:22:17.646 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-07T23:22:17.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.648 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-07T23:22:17.657 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-07T23:22:17.658 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:22:17.658 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-07T23:22:17.658 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:22:17.658 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:22:17.658 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-07T23:22:17.659 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-07T23:22:17.659 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:22:17.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.660 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:22:17.660 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:22:17.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:22:17.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:22:17.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:22:17.662 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:22:17.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.704 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:22:17.754 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T23:22:17.755 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T23:22:17.755 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:22:17.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:22:17.755 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-10-07T23:22:17.773 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T23:22:17.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:22:17.773 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:17.780 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:22:17.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:22:17.781 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:17.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:22:17.847 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:17.847 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:17.861 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:17.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:17.862 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:22:17.867 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:22:17.867 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T23:22:17.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:22:17.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:17.930 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:17.940 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:17.941 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:17.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:17.941 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:22:17.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2023-10-07T23:22:17.999+0000 7fd988148380 -1 init, newargv = 0x56144a5f6db0 newargc=15 2023-10-07T23:22:18.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39634]: starting ceph client 2023-10-07T23:22:18.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39634]: starting fuse 2023-10-07T23:22:18.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:22:18.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:22:19.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:22:19.106 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:22:19.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:22:19.165 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:22:19.166 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:22:19.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:19.166 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:22:19.173 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:22:19.173 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:22:19.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:19.173 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-10-07T23:22:19.174 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2023-10-07T23:22:19.175 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-10-07T23:22:19.256 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.39634.asok 2023-10-07T23:22:19.439 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39634.asok 2023-10-07T23:22:19.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:19.440 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39634.asok status 2023-10-07T23:22:19.581 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "39634", 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:22:19.582 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5134, 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5134 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3779124269 2023-10-07T23:22:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3779124269 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.5134 192.168.0.1:0/3779124269", 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3779124269", 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 13, 2023-10-07T23:22:19.584 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 35, 2023-10-07T23:22:19.585 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:22:19.585 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-10-07T23:22:19.585 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-10-07T23:22:19.585 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-10-07T23:22:19.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:19.591 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-10-07T23:22:19.591 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:19.598 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-10-07T23:22:19.599 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:19.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:19.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:22:19.612 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2023-10-07T23:22:19.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 Processor -- start 2023-10-07T23:22:19.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 -- start start 2023-10-07T23:22:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c06a900 con 0x7fe72c159030 2023-10-07T23:22:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c06a600 con 0x7fe72c06cf10 2023-10-07T23:22:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe732c36700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0b0450 con 0x7fe72c0cad10 2023-10-07T23:22:19.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.725+0000 7fe7309d2700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe72c159030 0x7fe72c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:47120/0 (socket says 172.21.15.57:47120) 2023-10-07T23:22:19.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7309d2700 1 -- 172.21.15.57:0/3693867884 learned_addr learned my addr 172.21.15.57:0/3693867884 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:22:19.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7311d3700 1 --2- 172.21.15.57:0/3693867884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939072650 0 0) 0x7fe72c06a900 con 0x7fe72c159030 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe714003570 con 0x7fe72c159030 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994190370 0 0) 0x7fe72c06a600 con 0x7fe72c06cf10 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe72c06a900 con 0x7fe72c06cf10 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7311d3700 1 -- 172.21.15.57:0/3693867884 >> v1:172.21.15.154:6789/0 conn(0x7fe72c06cf10 legacy=0x7fe72c0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7311d3700 1 -- 172.21.15.57:0/3693867884 >> v1:172.21.15.57:6789/0 conn(0x7fe72c159030 legacy=0x7fe72c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7311d3700 1 -- 172.21.15.57:0/3693867884 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0cb350 con 0x7fe72c0cad10 2023-10-07T23:22:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe7311d3700 1 --2- 172.21.15.57:0/3693867884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0aff10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe724009e90 tx=0x7fe724009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332e1670e0cf0776 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400e070 con 0x7fe72c0cad10 2023-10-07T23:22:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400bb80 con 0x7fe72c0cad10 2023-10-07T23:22:19.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe72b7fe700 1 -- 172.21.15.57:0/3693867884 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724012430 con 0x7fe72c0cad10 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 -- 172.21.15.57:0/3693867884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 msgr2=0x7fe72c0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 --2- 172.21.15.57:0/3693867884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0aff10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe724009e90 tx=0x7fe724009780 comp rx=0 tx=0).stop 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 -- 172.21.15.57:0/3693867884 shutdown_connections 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 --2- 172.21.15.57:0/3693867884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0aff10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 -- 172.21.15.57:0/3693867884 >> 172.21.15.57:0/3693867884 conn(0x7fe72c068650 msgr2=0x7fe72c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 -- 172.21.15.57:0/3693867884 shutdown_connections 2023-10-07T23:22:19.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.729+0000 7fe732c36700 1 -- 172.21.15.57:0/3693867884 wait complete. 2023-10-07T23:22:19.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.733+0000 7fe732c36700 1 Processor -- start 2023-10-07T23:22:19.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.733+0000 7fe732c36700 1 -- start start 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06cf10 0x7fe72c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c15a240 con 0x7fe72c159030 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c159630 con 0x7fe72c06cf10 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c159910 con 0x7fe72c0cad10 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41380/0 (socket says 172.21.15.57:41380) 2023-10-07T23:22:19.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 -- 172.21.15.57:0/1268868728 learned_addr learned my addr 172.21.15.57:0/1268868728 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:19.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7309d2700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06cf10 0x7fe72c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:19.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe72bfff700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 -- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 msgr2=0x7fe72c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 -- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06cf10 msgr2=0x7fe72c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06cf10 0x7fe72c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 -- 172.21.15.57:0/1268868728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009a20 con 0x7fe72c159030 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe7311d3700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0cfdb0 tx=0x7fe724019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724009420 con 0x7fe72c159030 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724027470 con 0x7fe72c159030 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c159bf0 con 0x7fe72c159030 2023-10-07T23:22:19.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724027810 con 0x7fe72c159030 2023-10-07T23:22:19.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c0c5940 con 0x7fe72c159030 2023-10-07T23:22:19.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe724024070 con 0x7fe72c159030 2023-10-07T23:22:19.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.737+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c15a3f0 con 0x7fe72c159030 2023-10-07T23:22:19.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.741+0000 7fe728ff9700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 0x7fe718062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:19.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.741+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe72407edd0 con 0x7fe72c159030 2023-10-07T23:22:19.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.741+0000 7fe7309d2700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 0x7fe718062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:19.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.741+0000 7fe7309d2700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 0x7fe718062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe720007fd0 tx=0x7fe720009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:19.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.745+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe72c15a3f0 con 0x7fe72c159030 2023-10-07T23:22:19.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.909+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) v1 -- 0x7fe72c0647b0 con 0x7fe72c159030 2023-10-07T23:22:19.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.913+0000 7fe728ff9700 1 -- 172.21.15.57:0/1268868728 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe72404f3c0 con 0x7fe72c159030 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 msgr2=0x7fe718062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 0x7fe718062a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe720007fd0 tx=0x7fe720009040 comp rx=0 tx=0).stop 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 msgr2=0x7fe72c0b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0cfdb0 tx=0x7fe724019000 comp rx=0 tx=0).stop 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 shutdown_connections 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe72c159030 0x7fe72c0b4630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe72c0cad10 0x7fe72c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe718060550 0x7fe718062a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 --2- 172.21.15.57:0/1268868728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe72c06cf10 0x7fe72c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:19.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 >> 172.21.15.57:0/1268868728 conn(0x7fe72c068650 msgr2=0x7fe72c0cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:19.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 shutdown_connections 2023-10-07T23:22:19.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:19.917+0000 7fe732c36700 1 -- 172.21.15.57:0/1268868728 wait complete. 2023-10-07T23:22:19.935 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2023-10-07T23:22:20.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.041+0000 7f66149bb700 1 Processor -- start 2023-10-07T23:22:20.048 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.041+0000 7f66149bb700 1 -- start start 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6610155610 0x7f66101559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f66101596f0 0x7f66100cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6610159c60 con 0x7f66100c9580 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6610159f40 con 0x7f6610155610 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661015a220 con 0x7f66101596f0 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6610155610 0x7f66101559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6610155610 0x7f66101559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41392/0 (socket says 172.21.15.57:41392) 2023-10-07T23:22:20.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 -- 172.21.15.57:0/1997374303 learned_addr learned my addr 172.21.15.57:0/1997374303 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:20.051 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660e59c700 1 --2- 172.21.15.57:0/1997374303 >> v2:172.21.15.154:3300/0 conn(0x7f66101596f0 0x7f66100cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 -- 172.21.15.57:0/1997374303 >> v2:172.21.15.154:3300/0 conn(0x7f66101596f0 msgr2=0x7f66100cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:22:20.052 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 --2- 172.21.15.57:0/1997374303 >> v2:172.21.15.154:3300/0 conn(0x7f66101596f0 0x7f66100cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 -- 172.21.15.57:0/1997374303 >> v1:172.21.15.154:6789/0 conn(0x7f66100c9580 legacy=0x7f6610156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 -- 172.21.15.57:0/1997374303 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661015a500 con 0x7f6610155610 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660ed9d700 1 --2- 172.21.15.57:0/1997374303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f66101559e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f661006c540 tx=0x7f6604009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a91f7ece0104e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660dd9b700 1 -- 172.21.15.57:0/1997374303 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604009e70 con 0x7f6610155610 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660dd9b700 1 -- 172.21.15.57:0/1997374303 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660400b860 con 0x7f6610155610 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f660dd9b700 1 -- 172.21.15.57:0/1997374303 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660400bc00 con 0x7f6610155610 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- 172.21.15.57:0/1997374303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 msgr2=0x7f66101559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 --2- 172.21.15.57:0/1997374303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f66101559e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f661006c540 tx=0x7f6604009250 comp rx=0 tx=0).stop 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- 172.21.15.57:0/1997374303 shutdown_connections 2023-10-07T23:22:20.053 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 --2- 172.21.15.57:0/1997374303 >> v2:172.21.15.154:3300/0 conn(0x7f66101596f0 0x7f66100cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 --2- 172.21.15.57:0/1997374303 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f66101559e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- 172.21.15.57:0/1997374303 >> 172.21.15.57:0/1997374303 conn(0x7f66100670e0 msgr2=0x7f66100674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- 172.21.15.57:0/1997374303 shutdown_connections 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.045+0000 7f66149bb700 1 -- 172.21.15.57:0/1997374303 wait complete. 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 Processor -- start 2023-10-07T23:22:20.054 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 -- start start 2023-10-07T23:22:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66100c9580 0x7f66100c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66101596f0 0x7f6610156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610167100 con 0x7f6610155610 2023-10-07T23:22:20.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c5f90 con 0x7f66101596f0 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f66149bb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100c6270 con 0x7f66100c9580 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41406/0 (socket says 172.21.15.57:41406) 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 -- 172.21.15.57:0/8205429 learned_addr learned my addr 172.21.15.57:0/8205429 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660ed9d700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66100c9580 0x7f66100c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.057 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660f59e700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66101596f0 0x7f6610156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 -- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66100c9580 msgr2=0x7f66100c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66100c9580 0x7f66100c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 -- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66101596f0 msgr2=0x7f6610156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66101596f0 0x7f6610156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 -- 172.21.15.57:0/8205429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040094f0 con 0x7f6610155610 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f660e59c700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f660000e930 tx=0x7f66000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.049+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600013680 con 0x7f6610155610 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66000137e0 con 0x7f6610155610 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66100c6550 con 0x7f6610155610 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600013b80 con 0x7f6610155610 2023-10-07T23:22:20.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100c3740 con 0x7f6610155610 2023-10-07T23:22:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66100711c0 con 0x7f6610155610 2023-10-07T23:22:20.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f660000c3e0 con 0x7f6610155610 2023-10-07T23:22:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f65f6ffd700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 0x7f65fc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:22:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.053+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f660007c860 con 0x7f6610155610 2023-10-07T23:22:20.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.057+0000 7f660ed9d700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 0x7f65fc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:22:20.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.057+0000 7f660ed9d700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 0x7f65fc062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f661006c540 tx=0x7f6604017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:22:20.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.061+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f660004ce50 con 0x7f6610155610 2023-10-07T23:22:20.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.217+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) v1 -- 0x7f66100ca9a0 con 0x7f6610155610 2023-10-07T23:22:20.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.229+0000 7f65f6ffd700 1 -- 172.21.15.57:0/8205429 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) v1 ==== 135+0+0 (secure 0 0 0) 0x7f66000170c0 con 0x7f6610155610 2023-10-07T23:22:20.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 msgr2=0x7f65fc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 0x7f65fc062a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f661006c540 tx=0x7f6604017000 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 msgr2=0x7f6610156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f660000e930 tx=0x7f66000091a0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 shutdown_connections 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f65fc060550 0x7f65fc062a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f66101596f0 0x7f6610156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6610155610 0x7f6610156620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 --2- 172.21.15.57:0/8205429 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f66100c9580 0x7f66100c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:22:20.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 >> 172.21.15.57:0/8205429 conn(0x7f66100670e0 msgr2=0x7f6610153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:22:20.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 shutdown_connections 2023-10-07T23:22:20.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:22:20.233+0000 7f66149bb700 1 -- 172.21.15.57:0/8205429 wait complete. 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> n = 25 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2023-10-07T23:22:20.255 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:22:20.256 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:22:20.257 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:20.714 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> n = 25 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2023-10-07T23:22:20.715 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:22:20.716 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:22:20.717 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:20.718 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:20.718 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:22:20.718 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:22:20.718 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:21.215 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:21.215 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> n = 25 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:22:21.216 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:22:21.217 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:22:21.218 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.219 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:22:21.220 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:22:21.221 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:21.679 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> n = 25 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:22:21.680 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.681 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:22:21.682 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:22.141 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:22:22.141 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:22:22.142 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-10-07T23:22:22.143 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:22:22.271 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.41926.asok 2023-10-07T23:22:22.458 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41926.asok 2023-10-07T23:22:22.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:22.458 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41926.asok mds_sessions 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5157, 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5157 2023-10-07T23:22:22.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 831190132 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5157 v1:192.168.0.1:0/831190132", 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "v1:192.168.0.1:0/831190132", 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions": [ 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": 0, 2023-10-07T23:22:22.555 INFO:teuthology.orchestra.run.smithi057.stdout: "addrs": { 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6834", 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2270357685 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6835", 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2270357685 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:22.556 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "seq": 204, 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_gen": 0, 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_ttl": "2023-10-07T23:23:16.670363+0000", 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "last_cap_renew_request": "2023-10-07T23:22:16.670363+0000", 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_renew_seq": 2, 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "num_caps": 105, 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "open" 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 13 2023-10-07T23:22:22.557 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:22:22.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:22:22.563 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'find . | wc') 2023-10-07T23:22:48.827 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:22:48.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:23:01.623 INFO:teuthology.orchestra.run.smithi057.stdout: 105 105 1490 2023-10-07T23:23:01.624 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.737+0000 7f62e759e700 1 Processor -- start 2023-10-07T23:23:01.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.737+0000 7f62e759e700 1 -- start start 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62e8158be0 0x7f62e80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62e806b850 0x7f62e80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e806b3c0 con 0x7f62e8158fb0 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e806a7b0 con 0x7f62e8158be0 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e806aa90 con 0x7f62e806b850 2023-10-07T23:23:01.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62e8158be0 0x7f62e80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f62e8158be0 0x7f62e80ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33552/0 (socket says 172.21.15.57:33552) 2023-10-07T23:23:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 -- 172.21.15.57:0/3798374933 learned_addr learned my addr 172.21.15.57:0/3798374933 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:01.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e6d9d700 1 --2- 172.21.15.57:0/3798374933 >> v2:172.21.15.154:3300/0 conn(0x7f62e806b850 0x7f62e80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e559a700 1 -- 172.21.15.57:0/3798374933 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513438484 0 0) 0x7f62e806b3c0 con 0x7f62e8158fb0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e559a700 1 -- 172.21.15.57:0/3798374933 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d4003570 con 0x7f62e8158fb0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 -- 172.21.15.57:0/3798374933 >> v2:172.21.15.154:3300/0 conn(0x7f62e806b850 msgr2=0x7f62e80d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 --2- 172.21.15.57:0/3798374933 >> v2:172.21.15.154:3300/0 conn(0x7f62e806b850 0x7f62e80d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 -- 172.21.15.57:0/3798374933 >> v1:172.21.15.154:6789/0 conn(0x7f62e8158fb0 legacy=0x7f62e80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 -- 172.21.15.57:0/3798374933 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e806ad70 con 0x7f62e8158be0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e659c700 1 --2- 172.21.15.57:0/3798374933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e80ce170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cf4c0 tx=0x7f62d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d871e41d58aa7789 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e559a700 1 -- 172.21.15.57:0/3798374933 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800b640 con 0x7f62e8158be0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e559a700 1 -- 172.21.15.57:0/3798374933 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d800b7a0 con 0x7f62e8158be0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e559a700 1 -- 172.21.15.57:0/3798374933 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800bb20 con 0x7f62e8158be0 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- 172.21.15.57:0/3798374933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 msgr2=0x7f62e80ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 --2- 172.21.15.57:0/3798374933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e80ce170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cf4c0 tx=0x7f62d8009250 comp rx=0 tx=0).stop 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- 172.21.15.57:0/3798374933 shutdown_connections 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 --2- 172.21.15.57:0/3798374933 >> v2:172.21.15.154:3300/0 conn(0x7f62e806b850 0x7f62e80d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 --2- 172.21.15.57:0/3798374933 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e80ce170 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- 172.21.15.57:0/3798374933 >> 172.21.15.57:0/3798374933 conn(0x7f62e8068520 msgr2=0x7f62e8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- 172.21.15.57:0/3798374933 shutdown_connections 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.741+0000 7f62e759e700 1 -- 172.21.15.57:0/3798374933 wait complete. 2023-10-07T23:23:01.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 Processor -- start 2023-10-07T23:23:01.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 -- start start 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 0x7f62e80c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e8158fb0 0x7f62e81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8157cd0 con 0x7f62e8158be0 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8157fb0 con 0x7f62e8158fb0 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8158290 con 0x7f62e806b850 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33554/0 (socket says 172.21.15.57:33554) 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 -- 172.21.15.57:0/3702818926 learned_addr learned my addr 172.21.15.57:0/3702818926 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e6d9d700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e8158fb0 0x7f62e81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e659c700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 0x7f62e80c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 -- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 msgr2=0x7f62e80c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 0x7f62e80c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 -- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e8158fb0 msgr2=0x7f62e81527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e8158fb0 0x7f62e81527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 -- 172.21.15.57:0/3702818926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80094f0 con 0x7f62e8158be0 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e659c700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 0x7f62e80c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:01.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.745+0000 7f62e5d9b700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c5b0 tx=0x7f62d0007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00105c0 con 0x7f62e8158be0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0010720 con 0x7f62e8158be0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e816a080 con 0x7f62e8158be0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0010ae0 con 0x7f62e8158be0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e80c99f0 con 0x7f62e8158be0 2023-10-07T23:23:01.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e816a580 con 0x7f62e8158be0 2023-10-07T23:23:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f62d00123e0 con 0x7f62e8158be0 2023-10-07T23:23:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 0x7f62c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:01.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f62d007c380 con 0x7f62e8158be0 2023-10-07T23:23:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.749+0000 7f62e659c700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 0x7f62c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:01.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.753+0000 7f62e659c700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 0x7f62c8062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cf4c0 tx=0x7f62d8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:01.760 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.757+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f62d004c970 con 0x7f62e8158be0 2023-10-07T23:23:01.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.949+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62e80711c0 con 0x7f62e8158be0 2023-10-07T23:23:01.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.949+0000 7f62de7fc700 1 -- 172.21.15.57:0/3702818926 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f62d004ff90 con 0x7f62e8158be0 2023-10-07T23:23:01.953 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:01.953 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:01.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 msgr2=0x7f62c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 0x7f62c8062a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f62e80cf4c0 tx=0x7f62d8017000 comp rx=0 tx=0).stop 2023-10-07T23:23:01.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 msgr2=0x7f62e8157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c5b0 tx=0x7f62d0007d90 comp rx=0 tx=0).stop 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 shutdown_connections 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f62c8060550 0x7f62c8062a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62e8158fb0 0x7f62e81527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62e8158be0 0x7f62e8157380 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 --2- 172.21.15.57:0/3702818926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f62e806b850 0x7f62e80c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 >> 172.21.15.57:0/3702818926 conn(0x7f62e8068520 msgr2=0x7f62e80688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 shutdown_connections 2023-10-07T23:23:01.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:01.953+0000 7f62e759e700 1 -- 172.21.15.57:0/3702818926 wait complete. 2023-10-07T23:23:01.957 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:01.965 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2023-10-07T23:23:02.071 INFO:teuthology.orchestra.run.smithi057.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":141,"msgr_send_messages":141,"msgr_recv_bytes":36091,"msgr_send_bytes":650140,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.022668770,"msgr_running_send_time":0.007661254,"msgr_running_recv_time":0.006350666,"msgr_running_fast_dispatch_time":0.005664851,"msgr_send_messages_queue_lat":{"avgcount":140,"sum":0.144545112,"avgtime":0.001032465},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"AsyncMessenger::Worker-1":{"msgr_recv_messages":492,"msgr_send_messages":474,"msgr_recv_bytes":146046,"msgr_send_bytes":346169,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.076109025,"msgr_running_send_time":0.032722480,"msgr_running_recv_time":0.052057432,"msgr_running_fast_dispatch_time":0.004055220,"msgr_send_messages_queue_lat":{"avgcount":472,"sum":0.055109048,"avgtime":0.000116756},"msgr_handle_ack_lat":{"avgcount":937,"sum":0.000303952,"avgtime":0.000000324}},"AsyncMessenger::Worker-2":{"msgr_recv_messages":131,"msgr_send_messages":80,"msgr_recv_bytes":127983,"msgr_send_bytes":29290,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.018806445,"msgr_running_send_time":0.004316794,"msgr_running_recv_time":0.012646818,"msgr_running_fast_dispatch_time":0.004583710,"msgr_send_messages_queue_lat":{"avgcount":80,"sum":0.069477316,"avgtime":0.000868466},"msgr_handle_ack_lat":{"avgcount":58,"sum":0.000014303,"avgtime":0.000000246}},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":254,"sum":0.121206377,"avgtime":0.000477190}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000015064,"avgtime":0.000007532}},"mds":{"request":130,"reply":130,"reply_latency":{"avgcount":130,"sum":39.133290491,"avgtime":0.301025311},"slow_reply":0,"forward":0,"dir_fetch":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":155,"traverse_hit":144,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":2307,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":9,"ceph_cap_op_grant":103,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":204,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":105},"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":312,"evex":0,"evtrm":0,"ev":312,"evexg":0,"evexd":0,"segadd":1,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4789012,"rdpos":0,"jlat":{"avgcount":111,"sum":0.392489199,"avgtime":0.003535938},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":114,"dn-":0,"cap":101,"cap+":106,"cap-":5,"rss":45720,"heap":207072},"mds_server":{"dispatch_client_request":217,"dispatch_server_request":0,"handle_client_request":130,"handle_client_session":110,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.095825935,"avgtime":0.000958259},"req_getattr_latency":{"avgcount":17,"sum":0.005675720,"avgtime":0.000333865},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":3,"sum":0.000773566,"avgtime":0.000257855},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.007652321,"avgtime":0.001913080},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":5,"sum":39.022838928,"avgtime":7.804567785},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000524021,"avgtime":0.000524021},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":82,"average_load":41,"avg_session_uptime":44,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_Buffer_bytes":0,"bluestore_Buffer_items":0,"bluestore_Extent_bytes":0,"bluestore_Extent_items":0,"bluestore_Blob_bytes":0,"bluestore_Blob_items":0,"bluestore_SharedBlob_bytes":0,"bluestore_SharedBlob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":222166,"buffer_anon_items":88,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5240,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":376288,"mds_co_items":2313,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":153,"op_send_bytes":613848,"op_resend":37,"op_reply":153,"oplen_avg":{"avgcount":153,"sum":181},"op":153,"op_r":0,"op_w":153,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":111,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":35,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":20,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":759,"get_sum":236892,"get_or_fail_fail":0,"get_or_fail_success":759,"take":0,"take_sum":0,"put":759,"put_sum":236892,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":613848,"put":153,"put_sum":613848,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":153,"put":153,"put_sum":153,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55c8354c7e00":{"val":0,"max":3758096384,"get_started":0,"get":312,"get_sum":594708,"get_or_fail_fail":0,"get_or_fail_success":312,"take":0,"take_sum":0,"put":111,"put_sum":594708,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-10-07T23:23:02.072 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 36091, "msgr_recv_messages": 141, "msgr_running_fast_dispatch_time": 0.005664851, "msgr_running_recv_time": 0.006350666, "msgr_running_send_time": 0.007661254, "msgr_running_total_time": 0.02266877, "msgr_send_bytes": 650140, "msgr_send_messages": 141, "msgr_send_messages_queue_lat": { "avgcount": 140, "avgtime": 0.001032465, "sum": 0.144545112 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 937, "avgtime": 3.24e-07, "sum": 0.000303952 }, "msgr_recv_bytes": 146046, "msgr_recv_messages": 492, "msgr_running_fast_dispatch_time": 0.00405522, "msgr_running_recv_time": 0.052057432, "msgr_running_send_time": 0.03272248, "msgr_running_total_time": 0.076109025, "msgr_send_bytes": 346169, "msgr_send_messages": 474, "msgr_send_messages_queue_lat": { "avgcount": 472, "avgtime": 0.000116756, "sum": 0.055109048 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 58, "avgtime": 2.46e-07, "sum": 1.4303e-05 }, "msgr_recv_bytes": 127983, "msgr_recv_messages": 131, "msgr_running_fast_dispatch_time": 0.00458371, "msgr_running_recv_time": 0.012646818, "msgr_running_send_time": 0.004316794, "msgr_running_total_time": 0.018806445, "msgr_send_bytes": 29290, "msgr_send_messages": 80, "msgr_send_messages_queue_lat": { "avgcount": 80, "avgtime": 0.000868466, "sum": 0.069477316 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 254, "avgtime": 0.00047719, "sum": 0.121206377 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 7.532e-06, "sum": 1.5064e-05 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 103, "ceph_cap_op_revoke": 9, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 204, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 2307, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 105, "q": 0, "reply": 130, "reply_latency": { "avgcount": 130, "avgtime": 0.301025311, "sum": 39.133290491 }, "request": 130, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 155, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 144, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 312, "evadd": 312, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 111, "avgtime": 0.003535938, "sum": 0.392489199 }, "rdpos": 0, "replayed": 0, "seg": 1, "segadd": 1, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4789012 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 114, "dn-": 0, "heap": 207072, "ino": 117, "ino+": 117, "ino-": 0, "rss": 45720 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 217, "dispatch_server_request": 0, "handle_client_request": 130, "handle_client_session": 110, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000958259, "sum": 0.095825935 }, "req_getattr_latency": { "avgcount": 17, "avgtime": 0.000333865, "sum": 0.00567572 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 3, "avgtime": 0.000257855, "sum": 0.000773566 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.00191308, "sum": 0.007652321 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.804567785, "sum": 39.022838928 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000524021, "sum": 0.000524021 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 41, "avg_session_uptime": 44, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 82 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_Blob_bytes": 0, "bluestore_Blob_items": 0, "bluestore_Buffer_bytes": 0, "bluestore_Buffer_items": 0, "bluestore_Extent_bytes": 0, "bluestore_Extent_items": 0, "bluestore_SharedBlob_bytes": 0, "bluestore_SharedBlob_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 222166, "buffer_anon_items": 88, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 376288, "mds_co_items": 2313, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5240, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 35, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 0, "omap_wr": 20, "op": 153, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 0, "op_reply": 153, "op_resend": 37, "op_rmw": 0, "op_send": 153, "op_send_bytes": 613848, "op_w": 153, "oplen_avg": { "avgcount": 153, "sum": 181 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 111, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 759, "get_or_fail_fail": 0, "get_or_fail_success": 759, "get_started": 0, "get_sum": 236892, "max": 104857600, "put": 759, "put_sum": 236892, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 153, "put_sum": 613848, "take": 153, "take_sum": 613848, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 153, "put_sum": 153, "take": 153, "take_sum": 153, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55c8354c7e00": { "get": 312, "get_or_fail_fail": 0, "get_or_fail_success": 312, "get_started": 0, "get_sum": 594708, "max": 3758096384, "put": 111, "put_sum": 594708, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-10-07T23:23:02.074 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:02.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.177+0000 7f783df71700 1 Processor -- start 2023-10-07T23:23:02.180 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.177+0000 7f783df71700 1 -- start start 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.177+0000 7f783df71700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7838158b00 0x7f7838068900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.177+0000 7f783df71700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 0x7f783806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783806d750 con 0x7f7838068530 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380c8b80 con 0x7f7838158b00 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380c7f70 con 0x7f78380c7800 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7838158b00 0x7f7838068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7838158b00 0x7f7838068900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33564/0 (socket says 172.21.15.57:33564) 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 -- 172.21.15.57:0/3284408828 learned_addr learned my addr 172.21.15.57:0/3284408828 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 -- 172.21.15.57:0/3284408828 >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 msgr2=0x7f783806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f7837fff700 1 --2- 172.21.15.57:0/3284408828 >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 0x7f783806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 --2- 172.21.15.57:0/3284408828 >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 0x7f783806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 -- 172.21.15.57:0/3284408828 >> v1:172.21.15.57:6789/0 conn(0x7f7838068530 legacy=0x7f78381583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 -- 172.21.15.57:0/3284408828 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380c8250 con 0x7f7838158b00 2023-10-07T23:23:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f7837fff700 1 --2- 172.21.15.57:0/3284408828 >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 0x7f783806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:23:02.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f782effd700 1 --2- 172.21.15.57:0/3284408828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838158b00 0x7f7838068900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f78380c6950 tx=0x7f7820009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bd13b7a1c05674 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f7836ffd700 1 -- 172.21.15.57:0/3284408828 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000b860 con 0x7f7838158b00 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f7836ffd700 1 -- 172.21.15.57:0/3284408828 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782000b9c0 con 0x7f7838158b00 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f7836ffd700 1 -- 172.21.15.57:0/3284408828 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000bd60 con 0x7f7838158b00 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- 172.21.15.57:0/3284408828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838158b00 msgr2=0x7f7838068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 --2- 172.21.15.57:0/3284408828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838158b00 0x7f7838068900 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f78380c6950 tx=0x7f7820009250 comp rx=0 tx=0).stop 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- 172.21.15.57:0/3284408828 shutdown_connections 2023-10-07T23:23:02.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 --2- 172.21.15.57:0/3284408828 >> v2:172.21.15.154:3300/0 conn(0x7f78380c7800 0x7f783806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 --2- 172.21.15.57:0/3284408828 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838158b00 0x7f7838068900 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- 172.21.15.57:0/3284408828 >> 172.21.15.57:0/3284408828 conn(0x7f78380cce30 msgr2=0x7f78380c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- 172.21.15.57:0/3284408828 shutdown_connections 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.181+0000 7f783df71700 1 -- 172.21.15.57:0/3284408828 wait complete. 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 Processor -- start 2023-10-07T23:23:02.185 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 -- start start 2023-10-07T23:23:02.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 0x7f78380afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 0x7f7838153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838156380 con 0x7f7838068530 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838155d70 con 0x7f78380c7800 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f783df71700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380b0560 con 0x7f7838158b00 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33570/0 (socket says 172.21.15.57:33570) 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 -- 172.21.15.57:0/1791413474 learned_addr learned my addr 172.21.15.57:0/1791413474 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f782effd700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 0x7f78380afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f7837fff700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 0x7f7838153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.189 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 -- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 msgr2=0x7f7838153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 0x7f7838153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 -- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 msgr2=0x7f78380afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 0x7f78380afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 -- 172.21.15.57:0/1791413474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200094f0 con 0x7f7838068530 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f7837fff700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 0x7f7838153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f782effd700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 0x7f78380afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:02.190 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f78377fe700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f782800c590 tx=0x7f782800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.185+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828011590 con 0x7f7838068530 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78280116f0 con 0x7f7838068530 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78380b0840 con 0x7f7838068530 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828011ab0 con 0x7f7838068530 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783814ea80 con 0x7f7838068530 2023-10-07T23:23:02.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78380b0a30 con 0x7f7838068530 2023-10-07T23:23:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f782801b3b0 con 0x7f7838068530 2023-10-07T23:23:02.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f782ffff700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 0x7f7818062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.189+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f782807bcb0 con 0x7f7838068530 2023-10-07T23:23:02.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.193+0000 7f782effd700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 0x7f7818062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.193+0000 7f782effd700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 0x7f7818062a00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78380c6950 tx=0x7f7820017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.201+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f782804c200 con 0x7f7838068530 2023-10-07T23:23:02.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.393+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f783806d750 con 0x7f7838068530 2023-10-07T23:23:02.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.393+0000 7f782ffff700 1 -- 172.21.15.57:0/1791413474 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f783806d750 con 0x7f7838068530 2023-10-07T23:23:02.397 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:02.397 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:02.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 msgr2=0x7f7818062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 0x7f7818062a00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78380c6950 tx=0x7f7820017000 comp rx=0 tx=0).stop 2023-10-07T23:23:02.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 msgr2=0x7f7838138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f782800c590 tx=0x7f782800c950 comp rx=0 tx=0).stop 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 shutdown_connections 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7838158b00 0x7f7838153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7818060550 0x7f7818062a00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78380c7800 0x7f78380afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 --2- 172.21.15.57:0/1791413474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7838068530 0x7f7838138c00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 >> 172.21.15.57:0/1791413474 conn(0x7f78380cce30 msgr2=0x7f7838152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 shutdown_connections 2023-10-07T23:23:02.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.397+0000 7f783df71700 1 -- 172.21.15.57:0/1791413474 wait complete. 2023-10-07T23:23:02.401 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:02.411 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-10-07T23:23:02.525 INFO:teuthology.orchestra.run.smithi057.stdout:[{"id":5134,"entity":{"name":{"type":"client","num":5134},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3779124269}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":44.515451835999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.7165456778983499,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5134 192.168.0.1:0/3779124269","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"1","hostname":"smithi154","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"39634","root":"/"}},{"id":5157,"entity":{"name":{"type":"client","num":5157},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":831190132}},"state":"open","num_leases":0,"num_caps":100,"request_load_avg":80,"uptime":46.847415357999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":6.3146035202564041,"halflife":60},"release_caps":{"value":3.212883839204046,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":294.93613023267721,"halflife":300},"cap_acquisition":{"value":60.504107493999371,"halflife":30},"last_trim_completed_requests_tid":128,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5157 192.168.0.1:0/831190132","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000017ffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"08cac0187919fc3a247efdf445f652e2fb30a647","ceph_version":"ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)","entity_id":"0","hostname":"smithi057","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"41926","root":"/"}}] 2023-10-07T23:23:02.525 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "1", "hostname": "smithi154", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "39634", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3779124269, "type": "any" }, "name": { "num": 5134, "type": "client" } }, "id": 5134, "inst": "client.5134 192.168.0.1:0/3779124269", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.71654567789835 }, "state": "open", "uptime": 44.515451836 }, { "cap_acquisition": { "halflife": 30, "value": 60.50410749399937 }, "client_metadata": { "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", "client_features": { "feature_bits": "0x000000000017ffff" }, "entity_id": "0", "hostname": "smithi057", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "41926", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 831190132, "type": "any" }, "name": { "num": 5157, "type": "client" } }, "id": 5157, "inst": "client.5157 192.168.0.1:0/831190132", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 128, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 6.314603520256404 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 3.212883839204046 }, "request_load_avg": 80, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 294.9361302326772 }, "state": "open", "uptime": 46.847415358 } ] 2023-10-07T23:23:02.526 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2023-10-07T23:23:02.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.633+0000 7f08ea818700 1 Processor -- start 2023-10-07T23:23:02.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.633+0000 7f08ea818700 1 -- start start 2023-10-07T23:23:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f08e4151090 0x7f08e415a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 0x7f08e40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e406ae80 con 0x7f08e40c8750 2023-10-07T23:23:02.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e40c8bd0 con 0x7f08e4151090 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e415b050 con 0x7f08e4157d90 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f08e4151090 0x7f08e415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f08e4151090 0x7f08e415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33582/0 (socket says 172.21.15.57:33582) 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 -- 172.21.15.57:0/1713473760 learned_addr learned my addr 172.21.15.57:0/1713473760 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9015700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 0x7f08e40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 -- 172.21.15.57:0/1713473760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 msgr2=0x7f08e40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 0x7f08e40c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 -- 172.21.15.57:0/1713473760 >> v1:172.21.15.154:6789/0 conn(0x7f08e40c8750 legacy=0x7f08e406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 -- 172.21.15.57:0/1713473760 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e415b330 con 0x7f08e4151090 2023-10-07T23:23:02.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9015700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 0x7f08e40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e9816700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4151090 0x7f08e415a750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009f20 tx=0x7f08e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92de7008c7a35fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e8814700 1 -- 172.21.15.57:0/1713473760 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e000e070 con 0x7f08e4151090 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e8814700 1 -- 172.21.15.57:0/1713473760 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e000ba80 con 0x7f08e4151090 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08e8814700 1 -- 172.21.15.57:0/1713473760 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00092e0 con 0x7f08e4151090 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- 172.21.15.57:0/1713473760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4151090 msgr2=0x7f08e415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4151090 0x7f08e415a750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009f20 tx=0x7f08e0009840 comp rx=0 tx=0).stop 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- 172.21.15.57:0/1713473760 shutdown_connections 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4157d90 0x7f08e40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 --2- 172.21.15.57:0/1713473760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4151090 0x7f08e415a750 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- 172.21.15.57:0/1713473760 >> 172.21.15.57:0/1713473760 conn(0x7f08e40670e0 msgr2=0x7f08e40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- 172.21.15.57:0/1713473760 shutdown_connections 2023-10-07T23:23:02.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.637+0000 7f08ea818700 1 -- 172.21.15.57:0/1713473760 wait complete. 2023-10-07T23:23:02.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 Processor -- start 2023-10-07T23:23:02.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- start start 2023-10-07T23:23:02.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08e40c8750 0x7f08e41464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4151090 0x7f08e4169690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41469f0 con 0x7f08e4157d90 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40b0d10 con 0x7f08e40c8750 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e40c94c0 con 0x7f08e4151090 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33596/0 (socket says 172.21.15.57:33596) 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 -- 172.21.15.57:0/1973667462 learned_addr learned my addr 172.21.15.57:0/1973667462 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08e9816700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08e40c8750 0x7f08e41464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08e9015700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4151090 0x7f08e4169690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 -- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4151090 msgr2=0x7f08e4169690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4151090 0x7f08e4169690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 -- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08e40c8750 msgr2=0x7f08e41464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08e40c8750 0x7f08e41464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 -- 172.21.15.57:0/1973667462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0009ae0 con 0x7f08e4157d90 2023-10-07T23:23:02.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea017700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c560 tx=0x7f08dc007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08d9ffb700 1 -- 172.21.15.57:0/1973667462 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011590 con 0x7f08e4157d90 2023-10-07T23:23:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08d9ffb700 1 -- 172.21.15.57:0/1973667462 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc0116f0 con 0x7f08e4157d90 2023-10-07T23:23:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e40c9d60 con 0x7f08e4157d90 2023-10-07T23:23:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08d9ffb700 1 -- 172.21.15.57:0/1973667462 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011ab0 con 0x7f08e4157d90 2023-10-07T23:23:02.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.641+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40c9710 con 0x7f08e4157d90 2023-10-07T23:23:02.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.645+0000 7f08d9ffb700 1 -- 172.21.15.57:0/1973667462 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f08dc01b3b0 con 0x7f08e4157d90 2023-10-07T23:23:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.645+0000 7f08d9ffb700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 0x7f08c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.645+0000 7f08d9ffb700 1 -- 172.21.15.57:0/1973667462 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08dc07bdc0 con 0x7f08e4157d90 2023-10-07T23:23:02.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.645+0000 7f08e9816700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 0x7f08c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08e9816700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 0x7f08c8062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08e41571e0 tx=0x7f08e0018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 Processor -- start 2023-10-07T23:23:02.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 -- start start 2023-10-07T23:23:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 0xecb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xdfcab0 con 0xc99850 2023-10-07T23:23:02.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xdfa080 con 0xc4e7d0 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xdf6020 con 0xc4eba0 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33612/0 (socket says 172.21.15.57:33612) 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 -- 172.21.15.57:0/1148426322 learned_addr learned my addr 172.21.15.57:0/1148426322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08e9816700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 0xecb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08e9015700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 -- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 msgr2=0xecb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 0xecb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 -- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 msgr2=0xecb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 -- 172.21.15.57:0/1148426322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc9aaa0 con 0xc99850 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08e9816700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 0xecb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ea017700 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a040 tx=0x7f08dc07a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b7e153a977846d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ceffd700 1 -- 172.21.15.57:0/1148426322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc050710 con 0xc99850 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ceffd700 1 -- 172.21.15.57:0/1148426322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc053260 con 0xc99850 2023-10-07T23:23:02.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.649+0000 7f08ceffd700 1 -- 172.21.15.57:0/1148426322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc056fb0 con 0xc99850 2023-10-07T23:23:02.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/1148426322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 msgr2=0xecbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00a040 tx=0x7f08dc07a1a0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/1148426322 shutdown_connections 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xecbe00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- 172.21.15.57:0/1148426322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4eba0 0xecb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/1148426322 >> 172.21.15.57:0/1148426322 conn(0xc4ef70 msgr2=0xe88a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/1148426322 shutdown_connections 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/1148426322 wait complete. 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- start start 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 Processor -- start 2023-10-07T23:23:02.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xecfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xe85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xdbc210 con 0xc4e7d0 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xdbbeb0 con 0xc4eba0 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xd94a10 con 0xc99850 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 10 client.0 _ll_register_callbacks cb 0xd98cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33614/0 (socket says 172.21.15.57:33614) 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 -- 172.21.15.57:0/3737498648 learned_addr learned my addr 172.21.15.57:0/3737498648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08ea017700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xe85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9015700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xecfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 -- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 msgr2=0xe85cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xe85cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 -- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 msgr2=0xecfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xecfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 -- 172.21.15.57:0/3737498648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e40c9710 con 0xc4e7d0 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08e9816700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009a30 tx=0x7f08e0020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a5b4b60162bbe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:23:02.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e001e500 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08e0024070 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xd98530 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e001e870 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 10 client.5208 fetch_fsmap learned FSMap version 13 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.653+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0xdc4410 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xd98530 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f08e001e9d0 con 0xc4e7d0 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 10 client.5208 fetch_fsmap finished waiting for FSMap version 13 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 10 client.5208 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-07T23:23:02.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 20 client.5208 populate_metadata read hostname 'smithi057' 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xed8b20 0xedaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 4 client.5208 mds_command: new command op to 4956 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 --> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xd98250 con 0xed8b20 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08e9015700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xed8b20 0xedaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 10 client.5208 ms_handle_connect on v2:172.21.15.57:6834/2270357685 2023-10-07T23:23:02.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08e9015700 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xed8b20 0xedaf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e235b83ee0d2f7bc server_cookie=31a605bfceef9fba in_seq=0 out_seq=0 2023-10-07T23:23:02.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 1 -- 172.21.15.57:0/3737498648 <== mds.0 v2:172.21.15.57:6834/2270357685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0xd98250 con 0xed8b20 2023-10-07T23:23:02.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 10 client.5208 handle_command_reply: tid=0 2023-10-07T23:23:02.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 client.5208 shutdown 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 msgr2=0xecf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009a30 tx=0x7f08e0020070 comp rx=0 tx=0).stop 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 shutdown_connections 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xed8b20 0xedaf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xe85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xecfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08c3fff700 0 client.5208 ms_handle_reset on v2:172.21.15.57:6834/2270357685 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 --2- 172.21.15.57:0/3737498648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4e7d0 0xecf990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 >> 172.21.15.57:0/3737498648 conn(0xc4ef70 msgr2=0xeb0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 shutdown_connections 2023-10-07T23:23:02.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 1 -- 172.21.15.57:0/3737498648 wait complete. 2023-10-07T23:23:02.666 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.657+0000 7f08edd23740 20 client.5208 trim_cache size 0 max 16384 2023-10-07T23:23:02.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 Processor -- start 2023-10-07T23:23:02.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 -- start start 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4e7d0 0xf20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xf20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xdc63a0 con 0xc99850 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xda1350 con 0xc4eba0 2023-10-07T23:23:02.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.669+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xdfb040 con 0xc4e7d0 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33616/0 (socket says 172.21.15.57:33616) 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 -- 172.21.15.57:0/3466748219 learned_addr learned my addr 172.21.15.57:0/3466748219 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08e9015700 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xf20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08e9816700 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4e7d0 0xf20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 -- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4e7d0 msgr2=0xf20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4e7d0 0xf20920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 -- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 msgr2=0xf20e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xf20e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 -- 172.21.15.57:0/3466748219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xdaae20 con 0xc99850 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08ea017700 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08dc04fcd0 tx=0x7f08dc063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbbf7fc7e3dcd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08cf7fe700 1 -- 172.21.15.57:0/3466748219 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc050710 con 0xc99850 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08cf7fe700 1 -- 172.21.15.57:0/3466748219 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc056fb0 con 0xc99850 2023-10-07T23:23:02.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08cf7fe700 1 -- 172.21.15.57:0/3466748219 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc06a130 con 0xc99850 2023-10-07T23:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 -- 172.21.15.57:0/3466748219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 msgr2=0xf213a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f08dc04fcd0 tx=0x7f08dc063000 comp rx=0 tx=0).stop 2023-10-07T23:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 -- 172.21.15.57:0/3466748219 shutdown_connections 2023-10-07T23:23:02.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc99850 0xf213a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4eba0 0xf20e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 --2- 172.21.15.57:0/3466748219 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc4e7d0 0xf20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 -- 172.21.15.57:0/3466748219 >> 172.21.15.57:0/3466748219 conn(0xc4ef70 msgr2=0xe87470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 -- 172.21.15.57:0/3466748219 shutdown_connections 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.673+0000 7f08edd23740 1 -- 172.21.15.57:0/3466748219 wait complete. 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 -- start start 2023-10-07T23:23:02.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 Processor -- start 2023-10-07T23:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xecbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xdf39e0 con 0xc4eba0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xdfa5e0 con 0xc4e7d0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xdcdc90 con 0xc99850 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08edd23740 10 client.0 _ll_register_callbacks cb 0xd973a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9816700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33620/0 (socket says 172.21.15.57:33620) 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08ea017700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xecbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 -- 172.21.15.57:0/924490400 learned_addr learned my addr 172.21.15.57:0/924490400 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9816700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:45780/0 (socket says 172.21.15.57:45780) 2023-10-07T23:23:02.681 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 -- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 msgr2=0xecbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xecbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 -- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 msgr2=0xecb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 -- 172.21.15.57:0/924490400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xc9aaa0 con 0xc4eba0 2023-10-07T23:23:02.682 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9816700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08e9015700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f08e40c37d0 tx=0x7f08d4018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1993178bac47fd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.677+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d401f440 con 0xc4eba0 2023-10-07T23:23:02.683 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08d401f5a0 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08dc053260 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d401f970 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 10 client.5214 fetch_fsmap learned FSMap version 13 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0xd9f1d0 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc053260 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 13) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f08d401fad0 con 0xc4eba0 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 10 client.5214 fetch_fsmap finished waiting for FSMap version 13 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 10 client.5214 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 20 client.5214 populate_metadata read hostname 'smithi057' 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xf2aba0 0xf2cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 4 client.5214 mds_command: new command op to 4956 tid=0[{"prefix": "dump_historic_ops"}] 2023-10-07T23:23:02.684 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 --> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] -- command(tid 0: {"prefix": "dump_historic_ops"}) v1 -- 0xca65a0 con 0xf2aba0 2023-10-07T23:23:02.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08e9816700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xf2aba0 0xf2cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:02.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 10 client.5214 ms_handle_connect on v2:172.21.15.57:6834/2270357685 2023-10-07T23:23:02.685 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08e9816700 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xf2aba0 0xf2cf80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ac45a557733bf4 server_cookie=4134f510f2ba63f in_seq=0 out_seq=0 2023-10-07T23:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 1 -- 172.21.15.57:0/924490400 <== mds.0 v2:172.21.15.57:6834/2270357685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69452 (crc 0 0 0) 0xca65a0 con 0xf2aba0 2023-10-07T23:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 10 client.5214 handle_command_reply: tid=0 2023-10-07T23:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 client.5214 shutdown 2023-10-07T23:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 msgr2=0xecb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f08e40c37d0 tx=0x7f08d4018040 comp rx=0 tx=0).stop 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 shutdown_connections 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:6834/2270357685,v1:172.21.15.57:6835/2270357685] conn(0xf2aba0 0xf2cf80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xc99850 0xecbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xc4eba0 0xecb9d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08c3fff700 0 client.5214 ms_handle_reset on v2:172.21.15.57:6834/2270357685 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 --2- 172.21.15.57:0/924490400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xc4e7d0 0xecb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 >> 172.21.15.57:0/924490400 conn(0xc4ef70 msgr2=0xe87d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.681+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 shutdown_connections 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08edd23740 1 -- 172.21.15.57:0/924490400 wait complete. 2023-10-07T23:23:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08edd23740 20 client.5214 trim_cache size 0 max 16384 2023-10-07T23:23:02.688 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:23:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "size": 20, 2023-10-07T23:23:02.688 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 600, 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "ops": [ 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:2 setattr mode=041777 #0x1 2023-10-07T23:22:17.242087+0000 caller_uid=0, caller_gid=0{0,})", 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:17.242304+0000", 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 45.441239883000001, 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.76639597199999998, 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:2", 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.689 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 2 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242304+0000", 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242305+0000", 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.690 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242304+0000", 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242323+0000", 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242427+0000", 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.691 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242728+0000", 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242835+0000", 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:17.242837+0000", 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.692 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:18.008349+0000", 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:18.008448+0000", 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:18.008643+0000", 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.693 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:18.008699+0000", 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:18.008700+0000", 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.694 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:5 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_0 2023-10-07T23:22:20.340306+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:20.340420+0000", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 42.343123525000003, 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.012684304, 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:5", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 5 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.695 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340420+0000", 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340422+0000", 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340420+0000", 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.696 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340437+0000", 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340478+0000", 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.340902+0000", 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.697 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.341199+0000", 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.341773+0000", 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.341774+0000", 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.698 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.352619+0000", 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.352907+0000", 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.353023+0000", 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.699 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.353104+0000", 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.353104+0000", 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.700 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:6 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_1 2023-10-07T23:22:20.354231+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:20.354450+0000", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 42.329093913999998, 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0092847880000000004, 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:6", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 6 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.701 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354450+0000", 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354451+0000", 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354450+0000", 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354465+0000", 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354524+0000", 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.354720+0000", 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.355314+0000", 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.355315+0000", 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.363136+0000", 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.363477+0000", 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.363638+0000", 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.363733+0000", 2023-10-07T23:23:02.705 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.363735+0000", 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:7 create owner_uid=1000, owner_gid=1267 #0x10000000000/file_2 2023-10-07T23:22:20.365037+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.706 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:20.365218+0000", 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 42.318325661000003, 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0087156839999999996, 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:7", 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 7 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.707 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365218+0000", 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365220+0000", 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365218+0000", 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.708 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365235+0000", 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365297+0000", 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.365562+0000", 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.709 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.366235+0000", 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.366235+0000", 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.373463+0000", 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.373758+0000", 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.373864+0000", 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.373933+0000", 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.373934+0000", 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:32 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir2 2023-10-07T23:22:20.840886+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:20.841003+0000", 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.842540276999998, 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.012016148000000001, 2023-10-07T23:23:02.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:32", 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 32 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841003+0000", 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841004+0000", 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841003+0000", 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841011+0000", 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841055+0000", 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.841779+0000", 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.842690+0000", 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.842692+0000", 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.852767+0000", 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.852903+0000", 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.852983+0000", 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853019+0000", 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853020+0000", 2023-10-07T23:23:02.717 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:33 create owner_uid=1000, owner_gid=1267 #0x1000000020e/file_0 2023-10-07T23:22:20.844168+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:20.844340+0000", 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.839203728999998, 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0089232740000000001, 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.718 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:33", 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 33 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844340+0000", 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.719 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844341+0000", 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844340+0000", 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844347+0000", 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.720 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844395+0000", 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.844867+0000", 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.845268+0000", 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.846120+0000", 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.846121+0000", 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853025+0000", 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853167+0000", 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853222+0000", 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853263+0000", 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:20.853263+0000", 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:60 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir3 2023-10-07T23:22:21.301420+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.301635+0000", 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.381908467000002, 2023-10-07T23:23:02.724 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.013135137, 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:60", 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 60 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301635+0000", 2023-10-07T23:23:02.725 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301636+0000", 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301635+0000", 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.726 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301647+0000", 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301676+0000", 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.301915+0000", 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.727 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.302191+0000", 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.302192+0000", 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314269+0000", 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314507+0000", 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314686+0000", 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314769+0000", 2023-10-07T23:23:02.729 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314770+0000", 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:61 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_0 2023-10-07T23:22:21.303513+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.730 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.303745+0000", 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.379798565999998, 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.011562688, 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:61", 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 61 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.731 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303745+0000", 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303747+0000", 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303745+0000", 2023-10-07T23:23:02.732 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303762+0000", 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303793+0000", 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.733 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.303966+0000", 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.304414+0000", 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.304721+0000", 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.304721+0000", 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.314776+0000", 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.315076+0000", 2023-10-07T23:23:02.735 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.315206+0000", 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.315305+0000", 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.736 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.315308+0000", 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:69 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_8 2023-10-07T23:22:21.370727+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.370871+0000", 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.312673115999999, 2023-10-07T23:23:02.737 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.011224361, 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:69", 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 69 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.738 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.370871+0000", 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.370872+0000", 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.370871+0000", 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.739 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.370883+0000", 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.370944+0000", 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.371168+0000", 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.371811+0000", 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.371812+0000", 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.381409+0000", 2023-10-07T23:23:02.741 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.381767+0000", 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.381957+0000", 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.382092+0000", 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.382095+0000", 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.743 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:70 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_9 2023-10-07T23:22:21.383362+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.383550+0000", 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.299993364999999, 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.01186517, 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:70", 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.744 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 70 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383550+0000", 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383552+0000", 2023-10-07T23:23:02.745 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383550+0000", 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383566+0000", 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.746 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383628+0000", 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.383902+0000", 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.384638+0000", 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.384639+0000", 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.394703+0000", 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.395061+0000", 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.395249+0000", 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.395412+0000", 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.395415+0000", 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.749 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:71 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_10 2023-10-07T23:22:21.396839+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.397061+0000", 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.286483136000001, 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0093933750000000007, 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.750 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:71", 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 71 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397061+0000", 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.751 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397062+0000", 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397061+0000", 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397077+0000", 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.752 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397141+0000", 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.397434+0000", 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.398355+0000", 2023-10-07T23:23:02.753 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.398356+0000", 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.405651+0000", 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.754 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.406040+0000", 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.406286+0000", 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.406451+0000", 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.755 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.406454+0000", 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:76 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_15 2023-10-07T23:22:21.439548+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.439713+0000", 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.243830643000003, 2023-10-07T23:23:02.756 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0091020070000000005, 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:76", 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 76 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.439713+0000", 2023-10-07T23:23:02.757 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.439714+0000", 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.439713+0000", 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.758 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.439726+0000", 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.439774+0000", 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.440019+0000", 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.759 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.440770+0000", 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.440771+0000", 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.448224+0000", 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.760 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.448579+0000", 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.448695+0000", 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.448814+0000", 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.448815+0000", 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:77 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_16 2023-10-07T23:22:21.450101+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.450308+0000", 2023-10-07T23:23:02.762 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.233235426999997, 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.010873433, 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:77", 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 77 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.763 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450308+0000", 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450309+0000", 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450308+0000", 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.764 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450321+0000", 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450372+0000", 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.450663+0000", 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.451515+0000", 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.451516+0000", 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.460176+0000", 2023-10-07T23:23:02.766 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.460752+0000", 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.461002+0000", 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.767 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.461178+0000", 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.461182+0000", 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.768 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:78 create owner_uid=1000, owner_gid=1267 #0x10000000228/file_17 2023-10-07T23:22:21.462535+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.462768+0000", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 41.220775691999997, 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.0099495620000000003, 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:78", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.769 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 78 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.462768+0000", 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.462769+0000", 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.770 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.462768+0000", 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.462783+0000", 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.462838+0000", 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.463104+0000", 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.463868+0000", 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.463869+0000", 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.772 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.471858+0000", 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.472341+0000", 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.472556+0000", 2023-10-07T23:23:02.773 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.472714+0000", 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.472718+0000", 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.774 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:88 mkdir owner_uid=1000, owner_gid=1267 #0x1/dir4 2023-10-07T23:22:21.763302+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.763404+0000", 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 40.920140119999999, 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.01295552, 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:88", 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.775 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 88 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763404+0000", 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.776 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763404+0000", 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763404+0000", 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763411+0000", 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763440+0000", 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763671+0000", 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.778 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763971+0000", 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.763972+0000", 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.775727+0000", 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.779 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776003+0000", 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776213+0000", 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776357+0000", 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.780 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776359+0000", 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:89 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_0 2023-10-07T23:22:21.765354+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.765523+0000", 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 40.918020413999997, 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.011596393, 2023-10-07T23:23:02.781 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:89", 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 89 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765523+0000", 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765524+0000", 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765523+0000", 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765530+0000", 2023-10-07T23:23:02.783 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765559+0000", 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.765787+0000", 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.766256+0000", 2023-10-07T23:23:02.784 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.766662+0000", 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.766663+0000", 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.785 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776367+0000", 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776713+0000", 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.776941+0000", 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.786 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.777112+0000", 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.777120+0000", 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.787 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:90 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_1 2023-10-07T23:22:21.778716+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.778931+0000", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 40.904613081000001, 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.011821824, 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:90", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 90 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.788 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.778931+0000", 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.778932+0000", 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.778931+0000", 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.778943+0000", 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.778978+0000", 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.779257+0000", 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.779974+0000", 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.779975+0000", 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.790395+0000", 2023-10-07T23:23:02.791 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.790605+0000", 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.790694+0000", 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.792 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.790752+0000", 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.790752+0000", 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:107 create owner_uid=1000, owner_gid=1267 #0x10000000242/file_18 2023-10-07T23:22:21.918322+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.793 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:21.918466+0000", 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 40.765077445999999, 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 0.010277537, 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:107", 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 107 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.794 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918466+0000", 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918467+0000", 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918466+0000", 2023-10-07T23:23:02.795 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918478+0000", 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918516+0000", 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.796 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.918717+0000", 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.919446+0000", 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "early_replied" 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.919447+0000", 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "submit entry: journal_and_reply" 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.797 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.928183+0000", 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "journal_committed: " 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.928529+0000", 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.928664+0000", 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.928743+0000", 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:21.928744+0000", 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:126 readdir #0x10000000242 2023-10-07T23:22:22.592484+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:22.592615+0000", 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 40.090928226999999, 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 16.508659222999999, 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:126", 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.800 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 126 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592615+0000", 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592616+0000", 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592615+0000", 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592625+0000", 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592652+0000", 2023-10-07T23:23:02.802 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:22.592717+0000", 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:23.092899+0000", 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.803 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:23.593076+0000", 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:24.093244+0000", 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:24.593444+0000", 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.804 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:25.093623+0000", 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:25.593812+0000", 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:26.093979+0000", 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.805 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:26.594147+0000", 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:27.094328+0000", 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:27.594507+0000", 2023-10-07T23:23:02.806 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:28.094700+0000", 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:28.594885+0000", 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:29.095089+0000", 2023-10-07T23:23:02.807 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:29.595292+0000", 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:30.095494+0000", 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:30.595723+0000", 2023-10-07T23:23:02.808 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:31.095903+0000", 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:31.596074+0000", 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.809 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:32.096265+0000", 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:32.596403+0000", 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:33.096601+0000", 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:33.596802+0000", 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:34.096990+0000", 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.811 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:34.597154+0000", 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:35.097325+0000", 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:35.597495+0000", 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.812 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:36.097694+0000", 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:36.597825+0000", 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:37.097955+0000", 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:37.598137+0000", 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:38.098326+0000", 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.814 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:38.598517+0000", 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.099071+0000", 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.100046+0000", 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.815 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.100894+0000", 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.100995+0000", 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.101271+0000", 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.816 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.101275+0000", 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "description": "client_request(client.5157:129 readdir #0x10000000228 2023-10-07T23:22:39.108621+0000 caller_uid=1000, caller_gid=1267{6,27,108,1267,10102,})", 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "initiated_at": "2023-10-07T23:22:39.108942+0000", 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "age": 23.574601977, 2023-10-07T23:23:02.817 INFO:teuthology.orchestra.run.smithi057.stdout: "duration": 22.510923866999999, 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type_data": { 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flag_point": "done", 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "reqid": "client.5157:129", 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "op_type": "client_request", 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "client_info": { 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "client": "client.5157", 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tid": 129 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "events": [ 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.818 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.108942+0000", 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "initiated" 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.108943+0000", 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "throttled" 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.108942+0000", 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "header_read" 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.819 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.108956+0000", 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "all_read" 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.109007+0000", 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "dispatched" 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.109151+0000", 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:39.609341+0000", 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:40.109509+0000", 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:40.609705+0000", 2023-10-07T23:23:02.821 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:41.109845+0000", 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:41.609977+0000", 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.822 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:42.110174+0000", 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:42.610350+0000", 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:43.110525+0000", 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.823 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:43.610712+0000", 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:44.110885+0000", 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:44.611060+0000", 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.824 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:45.111250+0000", 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:45.611419+0000", 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:46.111599+0000", 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.825 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:46.611785+0000", 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:47.111983+0000", 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:47.612177+0000", 2023-10-07T23:23:02.826 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:48.112352+0000", 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:48.612542+0000", 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.827 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:49.112723+0000", 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:49.612911+0000", 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:50.113094+0000", 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.828 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:50.613286+0000", 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:51.113484+0000", 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:51.613660+0000", 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.829 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:52.113790+0000", 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:52.613924+0000", 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.830 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 msgr2=0x7f08c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 0x7f08c8062a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f08e41571e0 tx=0x7f08e0018000 comp rx=0 tx=0).stop 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 msgr2=0x7f08e4159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c560 tx=0x7f08dc007ef0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:53.114133+0000", 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:53.614313+0000", 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:54.114437+0000", 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:54.614626+0000", 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.832 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:55.114820+0000", 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:55.615005+0000", 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:56.115153+0000", 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:56.615325+0000", 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:57.115519+0000", 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.834 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:57.615713+0000", 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:58.115903+0000", 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:58.616095+0000", 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.835 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:59.116284+0000", 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:22:59.616453+0000", 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:00.116630+0000", 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.836 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:00.616817+0000", 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.117029+0000", 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cap_acquisition_throttle" 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.617564+0000", 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "failed to rdlock, waiting" 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.837 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.618531+0000", 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "acquired locks" 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.619416+0000", 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "replying" 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.619521+0000", 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "finishing request" 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.838 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.619858+0000", 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "cleaned up request" 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "time": "2023-10-07T23:23:01.619866+0000", 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: "event": "done" 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:02.839 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:02.840 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:23:02.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 shutdown_connections 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f08e4157d90 0x7f08e4159540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f08e4151090 0x7f08e4169690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f08c8060550 0x7f08c8062a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 --2- 172.21.15.57:0/1973667462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f08e40c8750 0x7f08e41464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 >> 172.21.15.57:0/1973667462 conn(0x7f08e40670e0 msgr2=0x7f08e415a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 shutdown_connections 2023-10-07T23:23:02.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:02.685+0000 7f08ea818700 1 -- 172.21.15.57:0/1973667462 wait complete. 2023-10-07T23:23:02.842 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T23:23:02.842 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:23:02.862 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:23:02.883 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T23:23:02.883 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-07T23:23:02.883 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:23:02.883 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:02.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[41926]: fuse finished with error 0 and tester_r 0 2023-10-07T23:23:02.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:23:02.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:02.925 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T23:23:03.046 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-10-07T23:23:03.047 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:23:03.047 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:03.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39634]: fuse finished with error 0 and tester_r 0 2023-10-07T23:23:03.064 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:23:03.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:03.065 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T23:23:03.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:23:03.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:03.189 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T23:23:03.196 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-07T23:23:03.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:23:03.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:23:03.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:03.197 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T23:23:03.237 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-07T23:23:03.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:23:03.239 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:03.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 Processor -- start 2023-10-07T23:23:03.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 -- start start 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0afe90 0x7fec4c0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec4c15cb20 0x7fec4c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c0cb440 con 0x7fec4c0b0260 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c0cdbb0 con 0x7fec4c15cb20 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15cfb0 con 0x7fec4c0afe90 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4b7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fec4c0b0260 0x7fec4c159430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:53964/0 (socket says 172.21.15.57:53964) 2023-10-07T23:23:03.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4b7fe700 1 -- 172.21.15.57:0/4090009041 learned_addr learned my addr 172.21.15.57:0/4090009041 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:03.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/4090009041 >> v2:172.21.15.154:3300/0 conn(0x7fec4c15cb20 0x7fec4c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.353+0000 7fec4affd700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0afe90 0x7fec4c0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec49ffb700 1 -- 172.21.15.57:0/4090009041 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900410414 0 0) 0x7fec4c0cb440 con 0x7fec4c0b0260 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec49ffb700 1 -- 172.21.15.57:0/4090009041 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec34003570 con 0x7fec4c0b0260 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4a7fc700 1 -- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0afe90 msgr2=0x7fec4c0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0afe90 0x7fec4c0caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4a7fc700 1 -- 172.21.15.57:0/4090009041 >> v1:172.21.15.57:6789/0 conn(0x7fec4c0b0260 legacy=0x7fec4c159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4a7fc700 1 -- 172.21.15.57:0/4090009041 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c15d290 con 0x7fec4c15cb20 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c15cb20 0x7fec4c0cc030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e60 tx=0x7fec3c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb9e1716218ffdf server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec49ffb700 1 -- 172.21.15.57:0/4090009041 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00e070 con 0x7fec4c15cb20 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec49ffb700 1 -- 172.21.15.57:0/4090009041 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00bab0 con 0x7fec4c15cb20 2023-10-07T23:23:03.359 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec49ffb700 1 -- 172.21.15.57:0/4090009041 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00be70 con 0x7fec4c15cb20 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 -- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c15cb20 msgr2=0x7fec4c0cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c15cb20 0x7fec4c0cc030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e60 tx=0x7fec3c009780 comp rx=0 tx=0).stop 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 -- 172.21.15.57:0/4090009041 shutdown_connections 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c15cb20 0x7fec4c0cc030 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e60 tx=0x7fec3c009780 comp rx=0 tx=0).stop 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 --2- 172.21.15.57:0/4090009041 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c0afe90 0x7fec4c0caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 -- 172.21.15.57:0/4090009041 >> 172.21.15.57:0/4090009041 conn(0x7fec4c0692f0 msgr2=0x7fec4c0696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:03.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 -- 172.21.15.57:0/4090009041 shutdown_connections 2023-10-07T23:23:03.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.357+0000 7fec4bfff700 1 -- 172.21.15.57:0/4090009041 wait complete. 2023-10-07T23:23:03.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 Processor -- start 2023-10-07T23:23:03.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 -- start start 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0b0260 0x7fec4c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 0x7fec4c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15bca0 con 0x7fec4c0afe90 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c0b10d0 con 0x7fec4c0b0260 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15b090 con 0x7fec4c147990 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33638/0 (socket says 172.21.15.57:33638) 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 -- 172.21.15.57:0/3507477763 learned_addr learned my addr 172.21.15.57:0/3507477763 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4b7fe700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 0x7fec4c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0b0260 0x7fec4c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 -- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 msgr2=0x7fec4c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 0x7fec4c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 -- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0b0260 msgr2=0x7fec4c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0b0260 0x7fec4c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4affd700 1 -- 172.21.15.57:0/3507477763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009a20 con 0x7fec4c0afe90 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.361+0000 7fec4b7fe700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 0x7fec4c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec4affd700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec4000ec80 tx=0x7fec400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000c9d0 con 0x7fec4c0afe90 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4000cb30 con 0x7fec4c0afe90 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c15b370 con 0x7fec4c0afe90 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40013470 con 0x7fec4c0afe90 2023-10-07T23:23:03.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c15b930 con 0x7fec4c0afe90 2023-10-07T23:23:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c0711c0 con 0x7fec4c0afe90 2023-10-07T23:23:03.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fec40013820 con 0x7fec4c0afe90 2023-10-07T23:23:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 0x7fec38062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.365+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec4007cb10 con 0x7fec4c0afe90 2023-10-07T23:23:03.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.369+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 0x7fec38062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.369+0000 7fec4a7fc700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 0x7fec38062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06d940 tx=0x7fec3c019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.375 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.373+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fec4004d100 con 0x7fec4c0afe90 2023-10-07T23:23:03.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.569+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec4c15d570 con 0x7fec4c0afe90 2023-10-07T23:23:03.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.569+0000 7fec32ffd700 1 -- 172.21.15.57:0/3507477763 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fec40050720 con 0x7fec4c0afe90 2023-10-07T23:23:03.572 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:03.572 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 msgr2=0x7fec38062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 0x7fec38062a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06d940 tx=0x7fec3c019000 comp rx=0 tx=0).stop 2023-10-07T23:23:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 msgr2=0x7fec4c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec4000ec80 tx=0x7fec400091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 shutdown_connections 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fec4c147990 0x7fec4c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fec38060550 0x7fec38062a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec4c0b0260 0x7fec4c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 --2- 172.21.15.57:0/3507477763 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fec4c0afe90 0x7fec4c15aa20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 >> 172.21.15.57:0/3507477763 conn(0x7fec4c0692f0 msgr2=0x7fec4c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 shutdown_connections 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.573+0000 7fec4bfff700 1 -- 172.21.15.57:0/3507477763 wait complete. 2023-10-07T23:23:03.575 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:03.590 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:03.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.693+0000 7f59b1ab4700 1 Processor -- start 2023-10-07T23:23:03.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.693+0000 7f59b1ab4700 1 -- start start 2023-10-07T23:23:03.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59ac151090 0x7f59ac15a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac06ae80 con 0x7f59ac157d90 2023-10-07T23:23:03.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac15b050 con 0x7f59ac151090 2023-10-07T23:23:03.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac15b330 con 0x7f59ac0c8920 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:50506/0 (socket says 172.21.15.57:50506) 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 -- 172.21.15.57:0/3871729273 learned_addr learned my addr 172.21.15.57:0/3871729273 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59abfff700 1 --1- 172.21.15.57:0/3871729273 >> v1:172.21.15.154:6789/0 conn(0x7f59ac157d90 0x7f59ac0c8210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:51876/0 (socket says 172.21.15.57:51876) 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b0ab2700 1 --2- 172.21.15.57:0/3871729273 >> v2:172.21.15.154:3300/0 conn(0x7f59ac151090 0x7f59ac15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 -- 172.21.15.57:0/3871729273 >> v2:172.21.15.154:3300/0 conn(0x7f59ac151090 msgr2=0x7f59ac15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 --2- 172.21.15.57:0/3871729273 >> v2:172.21.15.154:3300/0 conn(0x7f59ac151090 0x7f59ac15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59ab7fe700 1 -- 172.21.15.57:0/3871729273 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528213115 0 0) 0x7f59ac06ae80 con 0x7f59ac157d90 2023-10-07T23:23:03.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 -- 172.21.15.57:0/3871729273 >> v1:172.21.15.154:6789/0 conn(0x7f59ac157d90 legacy=0x7f59ac0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 -- 172.21.15.57:0/3871729273 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac06ab10 con 0x7f59ac0c8920 2023-10-07T23:23:03.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b12b3700 1 --2- 172.21.15.57:0/3871729273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009f20 tx=0x7f59a4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ac855ad41d904c server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59ab7fe700 1 -- 172.21.15.57:0/3871729273 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400e070 con 0x7f59ac0c8920 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59ab7fe700 1 -- 172.21.15.57:0/3871729273 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400b9d0 con 0x7f59ac0c8920 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59ab7fe700 1 -- 172.21.15.57:0/3871729273 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40092e0 con 0x7f59ac0c8920 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3871729273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 msgr2=0x7f59ac06b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3871729273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009f20 tx=0x7f59a4009840 comp rx=0 tx=0).stop 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3871729273 shutdown_connections 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3871729273 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac0c8920 0x7f59ac06b310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3871729273 >> v2:172.21.15.154:3300/0 conn(0x7f59ac151090 0x7f59ac15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3871729273 >> 172.21.15.57:0/3871729273 conn(0x7f59ac0670e0 msgr2=0x7f59ac0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:03.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3871729273 shutdown_connections 2023-10-07T23:23:03.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.697+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3871729273 wait complete. 2023-10-07T23:23:03.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 Processor -- start 2023-10-07T23:23:03.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- start start 2023-10-07T23:23:03.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59ac0c8920 0x7f59ac0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ac151090 0x7f59ac0c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0c3e00 con 0x7f59ac0c8920 2023-10-07T23:23:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cab30 con 0x7f59ac151090 2023-10-07T23:23:03.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0c3f90 con 0x7f59ac157d90 2023-10-07T23:23:03.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b0ab2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59ac0c8920 0x7f59ac0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:50508/0 (socket says 172.21.15.57:50508) 2023-10-07T23:23:03.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 -- 172.21.15.57:0/3720009017 learned_addr learned my addr 172.21.15.57:0/3720009017 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:03.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59abfff700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ac151090 0x7f59ac0c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 -- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ac151090 msgr2=0x7f59ac0c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ac151090 0x7f59ac0c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 -- 172.21.15.57:0/3720009017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59ac0c8920 msgr2=0x7f59ac0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59ac0c8920 0x7f59ac0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 -- 172.21.15.57:0/3720009017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4009ae0 con 0x7f59ac157d90 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b12b3700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59ac15b470 tx=0x7f59a4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4008b10 con 0x7f59ac157d90 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4008c90 con 0x7f59ac157d90 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac0cad60 con 0x7f59ac157d90 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40266b0 con 0x7f59ac157d90 2023-10-07T23:23:03.705 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.701+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac0cb0f0 con 0x7f59ac157d90 2023-10-07T23:23:03.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac0711c0 con 0x7f59ac157d90 2023-10-07T23:23:03.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f59a4003040 con 0x7f59ac157d90 2023-10-07T23:23:03.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59a8ff9700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 0x7f5998062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:03.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59a407ebd0 con 0x7f59ac157d90 2023-10-07T23:23:03.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59b0ab2700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 0x7f5998062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:03.708 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.705+0000 7f59b0ab2700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 0x7f5998062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009db0 tx=0x7f59a00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:03.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.717+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f59a404f1a0 con 0x7f59ac157d90 2023-10-07T23:23:03.881 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.877+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ac068780 con 0x7f59ac157d90 2023-10-07T23:23:03.883 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.881+0000 7f59a8ff9700 1 -- 172.21.15.57:0/3720009017 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f59a4011e60 con 0x7f59ac157d90 2023-10-07T23:23:03.883 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:03.883 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:10.194488+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 msgr2=0x7f5998062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 0x7f5998062a00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009db0 tx=0x7f59a00091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 msgr2=0x7f59ac14f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f59ac15b470 tx=0x7f59a4018000 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 shutdown_connections 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f59ac157d90 0x7f59ac14f670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5998060550 0x7f5998062a00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59ac151090 0x7f59ac0c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 --2- 172.21.15.57:0/3720009017 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f59ac0c8920 0x7f59ac0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:03.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 >> 172.21.15.57:0/3720009017 conn(0x7f59ac0670e0 msgr2=0x7f59ac150570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:03.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 shutdown_connections 2023-10-07T23:23:03.887 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:03.885+0000 7f59b1ab4700 1 -- 172.21.15.57:0/3720009017 wait complete. 2023-10-07T23:23:03.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:22:06.862334+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:22:07.528301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:23:03.899 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:23:04.011 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 Processor -- start 2023-10-07T23:23:04.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 -- start start 2023-10-07T23:23:04.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b4006a6f0 0x7f3b4006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b400ca530 con 0x7f3b4015d580 2023-10-07T23:23:04.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4015d9f0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.009+0000 7f3b45726700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4006ae60 con 0x7f3b40158c40 2023-10-07T23:23:04.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3b4006a6f0 0x7f3b4006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3f7fe700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3b4015d580 0x7f3b400afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:51884/0 (socket says 172.21.15.57:51884) 2023-10-07T23:23:04.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3f7fe700 1 -- 172.21.15.57:0/666545538 learned_addr learned my addr 172.21.15.57:0/666545538 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:04.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 -- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 msgr2=0x7f3b40159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:04.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 --2- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 -- 172.21.15.57:0/666545538 >> v1:172.21.15.154:6789/0 conn(0x7f3b4015d580 legacy=0x7f3b400afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3effd700 1 -- 172.21.15.57:0/666545538 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389596819 0 0) 0x7f3b400ca530 con 0x7f3b4015d580 2023-10-07T23:23:04.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 -- 172.21.15.57:0/666545538 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4006b140 con 0x7f3b4006a6f0 2023-10-07T23:23:04.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b44f25700 1 --2- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4006a6f0 0x7f3b4006c800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009e60 tx=0x7f3b38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e42206bf5b5fa17 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3effd700 1 -- 172.21.15.57:0/666545538 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800e070 con 0x7f3b4006a6f0 2023-10-07T23:23:04.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3effd700 1 -- 172.21.15.57:0/666545538 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3800bb60 con 0x7f3b4006a6f0 2023-10-07T23:23:04.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b3effd700 1 -- 172.21.15.57:0/666545538 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38012430 con 0x7f3b4006a6f0 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 -- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4006a6f0 msgr2=0x7f3b4006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 --2- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4006a6f0 0x7f3b4006c800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009e60 tx=0x7f3b38009780 comp rx=0 tx=0).stop 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 -- 172.21.15.57:0/666545538 shutdown_connections 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 --2- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4006a6f0 0x7f3b4006c800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 --2- 172.21.15.57:0/666545538 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 -- 172.21.15.57:0/666545538 >> 172.21.15.57:0/666545538 conn(0x7f3b40062ba0 msgr2=0x7f3b40062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 -- 172.21.15.57:0/666545538 shutdown_connections 2023-10-07T23:23:04.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.013+0000 7f3b45726700 1 -- 172.21.15.57:0/666545538 wait complete. 2023-10-07T23:23:04.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 Processor -- start 2023-10-07T23:23:04.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- start start 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4015d580 0x7f3b400c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b400cdc60 con 0x7f3b4006a6f0 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b400cd670 con 0x7f3b4015d580 2023-10-07T23:23:04.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4015aa10 con 0x7f3b40158c40 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33646/0 (socket says 172.21.15.57:33646) 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 -- 172.21.15.57:0/901250043 learned_addr learned my addr 172.21.15.57:0/901250043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3f7fe700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b44f25700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4015d580 0x7f3b400c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 -- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 msgr2=0x7f3b40159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 -- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4015d580 msgr2=0x7f3b400c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4015d580 0x7f3b400c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 -- 172.21.15.57:0/901250043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b38009a20 con 0x7f3b4006a6f0 2023-10-07T23:23:04.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b3ffff700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800ecb0 tx=0x7f3b280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800c850 con 0x7f3b4006a6f0 2023-10-07T23:23:04.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2800c9b0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4015acf0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800cd50 con 0x7f3b4006a6f0 2023-10-07T23:23:04.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.017+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4015bb10 con 0x7f3b4006a6f0 2023-10-07T23:23:04.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.021+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4015af30 con 0x7f3b4006a6f0 2023-10-07T23:23:04.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.021+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3b28013680 con 0x7f3b4006a6f0 2023-10-07T23:23:04.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.021+0000 7f3b37fff700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 0x7f3b20062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.021+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b2807c9f0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.021+0000 7f3b3f7fe700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 0x7f3b20062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.025+0000 7f3b3f7fe700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 0x7f3b20062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3b400ca470 tx=0x7f3b30007f50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.029+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3b4015af30 con 0x7f3b4006a6f0 2023-10-07T23:23:04.191 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.189+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b4015aee0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.189+0000 7f3b37fff700 1 -- 172.21.15.57:0/901250043 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b2804cfe0 con 0x7f3b4006a6f0 2023-10-07T23:23:04.192 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 msgr2=0x7f3b20062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 0x7f3b20062a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3b400ca470 tx=0x7f3b30007f50 comp rx=0 tx=0).stop 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 msgr2=0x7f3b401548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800ecb0 tx=0x7f3b280091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 shutdown_connections 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3b20060550 0x7f3b20062a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b4015d580 0x7f3b400c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3b40158c40 0x7f3b40159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 --2- 172.21.15.57:0/901250043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3b4006a6f0 0x7f3b401548e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 >> 172.21.15.57:0/901250043 conn(0x7f3b40062ba0 msgr2=0x7f3b400b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 shutdown_connections 2023-10-07T23:23:04.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.193+0000 7f3b45726700 1 -- 172.21.15.57:0/901250043 wait complete. 2023-10-07T23:23:04.207 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:23:04.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.313+0000 7f2a44d32700 1 Processor -- start 2023-10-07T23:23:04.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.313+0000 7f2a44d32700 1 -- start start 2023-10-07T23:23:04.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.313+0000 7f2a44d32700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2a40159030 0x7f2a400aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.313+0000 7f2a44d32700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a4006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.313+0000 7f2a44d32700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4006a900 con 0x7f2a400ca600 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a400b0450 con 0x7f2a40159030 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4006a600 con 0x7f2a4006c800 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2a40159030 0x7f2a400aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2a40159030 0x7f2a400aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33654/0 (socket says 172.21.15.57:33654) 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 -- 172.21.15.57:0/2163198669 learned_addr learned my addr 172.21.15.57:0/2163198669 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3dd9b700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a4006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 -- 172.21.15.57:0/2163198669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 msgr2=0x7f2a4006cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a4006cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 -- 172.21.15.57:0/2163198669 >> v1:172.21.15.154:6789/0 conn(0x7f2a400ca600 legacy=0x7f2a400caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3d59a700 1 -- 172.21.15.57:0/2163198669 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341990076 0 0) 0x7f2a4006a900 con 0x7f2a400ca600 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 -- 172.21.15.57:0/2163198669 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4006d170 con 0x7f2a40159030 2023-10-07T23:23:04.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3e59c700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400aff10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a340099b0 tx=0x7f2a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57accc7cc4c5128 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3d59a700 1 -- 172.21.15.57:0/2163198669 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34009ea0 con 0x7f2a40159030 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3d59a700 1 -- 172.21.15.57:0/2163198669 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3400b860 con 0x7f2a40159030 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a3d59a700 1 -- 172.21.15.57:0/2163198669 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400bc00 con 0x7f2a40159030 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- 172.21.15.57:0/2163198669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 msgr2=0x7f2a400aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400aff10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a340099b0 tx=0x7f2a34009250 comp rx=0 tx=0).stop 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- 172.21.15.57:0/2163198669 shutdown_connections 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a4006cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 --2- 172.21.15.57:0/2163198669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400aff10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- 172.21.15.57:0/2163198669 >> 172.21.15.57:0/2163198669 conn(0x7f2a40068650 msgr2=0x7f2a40068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- 172.21.15.57:0/2163198669 shutdown_connections 2023-10-07T23:23:04.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- 172.21.15.57:0/2163198669 wait complete. 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 Processor -- start 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.317+0000 7f2a44d32700 1 -- start start 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a400b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a400ca600 0x7f2a401596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40159bf0 con 0x7f2a40159030 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4015a9f0 con 0x7f2a400ca600 2023-10-07T23:23:04.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40159de0 con 0x7f2a4006c800 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a400b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33656/0 (socket says 172.21.15.57:33656) 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 -- 172.21.15.57:0/1417785967 learned_addr learned my addr 172.21.15.57:0/1417785967 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3dd9b700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a400ca600 0x7f2a401596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 -- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 msgr2=0x7f2a400b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a400b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 -- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a400ca600 msgr2=0x7f2a401596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a400ca600 0x7f2a401596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 -- 172.21.15.57:0/1417785967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340094f0 con 0x7f2a40159030 2023-10-07T23:23:04.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a3ed9d700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001560 tx=0x7f2a38001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004160 con 0x7f2a40159030 2023-10-07T23:23:04.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3800a7e0 con 0x7f2a40159030 2023-10-07T23:23:04.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4015a0c0 con 0x7f2a40159030 2023-10-07T23:23:04.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800aba0 con 0x7f2a40159030 2023-10-07T23:23:04.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4015a680 con 0x7f2a40159030 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.321+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a400647b0 con 0x7f2a40159030 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.325+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2a38012420 con 0x7f2a40159030 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.325+0000 7f2a2e7fc700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 0x7f2a20062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.325+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a3807c590 con 0x7f2a40159030 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.325+0000 7f2a3e59c700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 0x7f2a20062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.325+0000 7f2a3e59c700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 0x7f2a20062a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a340099b0 tx=0x7f2a34017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.329+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2a3804cb80 con 0x7f2a40159030 2023-10-07T23:23:04.498 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.497+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a400cb900 con 0x7f2a40159030 2023-10-07T23:23:04.499 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.497+0000 7f2a2e7fc700 1 -- 172.21.15.57:0/1417785967 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a380501a0 con 0x7f2a40159030 2023-10-07T23:23:04.499 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:23:04.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 msgr2=0x7f2a20062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 0x7f2a20062a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a340099b0 tx=0x7f2a34017000 comp rx=0 tx=0).stop 2023-10-07T23:23:04.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 msgr2=0x7f2a400c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001560 tx=0x7f2a38001770 comp rx=0 tx=0).stop 2023-10-07T23:23:04.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 shutdown_connections 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2a40159030 0x7f2a400c5e70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a20060580 0x7f2a20062a30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a400ca600 0x7f2a401596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 --2- 172.21.15.57:0/1417785967 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2a4006c800 0x7f2a400b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 >> 172.21.15.57:0/1417785967 conn(0x7f2a40068650 msgr2=0x7f2a400ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 shutdown_connections 2023-10-07T23:23:04.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.501+0000 7f2a44d32700 1 -- 172.21.15.57:0/1417785967 wait complete. 2023-10-07T23:23:04.521 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:23:04.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 Processor -- start 2023-10-07T23:23:04.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 -- start start 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb55406cf10 0x7fb5540aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb5540ca600 0x7fb5540caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55406a900 con 0x7fb554159030 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540b0450 con 0x7fb5540ca600 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5599b5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55406a600 con 0x7fb55406cf10 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5537fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb5540ca600 0x7fb5540caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5537fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb5540ca600 0x7fb5540caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33658/0 (socket says 172.21.15.57:33658) 2023-10-07T23:23:04.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5537fe700 1 -- 172.21.15.57:0/3644346959 learned_addr learned my addr 172.21.15.57:0/3644346959 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.633+0000 7fb5527fc700 1 --2- 172.21.15.57:0/3644346959 >> v2:172.21.15.154:3300/0 conn(0x7fb55406cf10 0x7fb5540aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5537fe700 1 -- 172.21.15.57:0/3644346959 >> v2:172.21.15.154:3300/0 conn(0x7fb55406cf10 msgr2=0x7fb5540aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5537fe700 1 --2- 172.21.15.57:0/3644346959 >> v2:172.21.15.154:3300/0 conn(0x7fb55406cf10 0x7fb5540aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5537fe700 1 -- 172.21.15.57:0/3644346959 >> v1:172.21.15.57:6789/0 conn(0x7fb554159030 legacy=0x7fb55406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5537fe700 1 -- 172.21.15.57:0/3644346959 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540cb350 con 0x7fb5540ca600 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5537fe700 1 --2- 172.21.15.57:0/3644346959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb5540caa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ca540 tx=0x7fb54c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259a66d46140df26 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb551ffb700 1 -- 172.21.15.57:0/3644346959 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00e070 con 0x7fb5540ca600 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb551ffb700 1 -- 172.21.15.57:0/3644346959 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00ba50 con 0x7fb5540ca600 2023-10-07T23:23:04.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb551ffb700 1 -- 172.21.15.57:0/3644346959 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0092e0 con 0x7fb5540ca600 2023-10-07T23:23:04.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- 172.21.15.57:0/3644346959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 msgr2=0x7fb5540caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 --2- 172.21.15.57:0/3644346959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb5540caa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ca540 tx=0x7fb54c009840 comp rx=0 tx=0).stop 2023-10-07T23:23:04.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- 172.21.15.57:0/3644346959 shutdown_connections 2023-10-07T23:23:04.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 --2- 172.21.15.57:0/3644346959 >> v2:172.21.15.154:3300/0 conn(0x7fb55406cf10 0x7fb5540aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 --2- 172.21.15.57:0/3644346959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb5540caa80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- 172.21.15.57:0/3644346959 >> 172.21.15.57:0/3644346959 conn(0x7fb554068650 msgr2=0x7fb554068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- 172.21.15.57:0/3644346959 shutdown_connections 2023-10-07T23:23:04.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- 172.21.15.57:0/3644346959 wait complete. 2023-10-07T23:23:04.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 Processor -- start 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.637+0000 7fb5599b5700 1 -- start start 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb55406cf10 0x7fb55415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 0x7fb5540ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415bb30 con 0x7fb5540ca600 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415af20 con 0x7fb55406cf10 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415b200 con 0x7fb554159030 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.643 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33672/0 (socket says 172.21.15.57:33672) 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 -- 172.21.15.57:0/2364040128 learned_addr learned my addr 172.21.15.57:0/2364040128 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb552ffd700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb55406cf10 0x7fb55415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5537fe700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 0x7fb5540ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 -- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 msgr2=0x7fb5540ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 0x7fb5540ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 -- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb55406cf10 msgr2=0x7fb55415a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb55406cf10 0x7fb55415a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 -- 172.21.15.57:0/2364040128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009ae0 con 0x7fb5540ca600 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5537fe700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 0x7fb5540ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:04.644 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5527fc700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb54400d890 tx=0x7fb544008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440125f0 con 0x7fb5540ca600 2023-10-07T23:23:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544012770 con 0x7fb5540ca600 2023-10-07T23:23:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55415b4e0 con 0x7fb5540ca600 2023-10-07T23:23:04.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544012b30 con 0x7fb5540ca600 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554155030 con 0x7fb5540ca600 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55415bce0 con 0x7fb5540ca600 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.641+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb54401e460 con 0x7fb5540ca600 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.645+0000 7fb53affd700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 0x7fb540062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.645+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb54407ce00 con 0x7fb5540ca600 2023-10-07T23:23:04.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.645+0000 7fb552ffd700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 0x7fb540062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.645+0000 7fb552ffd700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 0x7fb540062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5480039e0 tx=0x7fb548007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.653+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb55415bce0 con 0x7fb5540ca600 2023-10-07T23:23:04.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.817+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5540647b0 con 0x7fb5540ca600 2023-10-07T23:23:04.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.817+0000 7fb53affd700 1 -- 172.21.15.57:0/2364040128 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb54404d3f0 con 0x7fb5540ca600 2023-10-07T23:23:04.820 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 msgr2=0x7fb540062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 0x7fb540062a00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5480039e0 tx=0x7fb548007ec0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 msgr2=0x7fb554152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb54400d890 tx=0x7fb544008040 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 shutdown_connections 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb554159030 0x7fb5540ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5540ca600 0x7fb554152380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb540060550 0x7fb540062a00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 --2- 172.21.15.57:0/2364040128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb55406cf10 0x7fb55415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 >> 172.21.15.57:0/2364040128 conn(0x7fb554068650 msgr2=0x7fb5540b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 shutdown_connections 2023-10-07T23:23:04.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.821+0000 7fb5599b5700 1 -- 172.21.15.57:0/2364040128 wait complete. 2023-10-07T23:23:04.835 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:04.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 Processor -- start 2023-10-07T23:23:04.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 -- start start 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe0ec15ced0 0x7fe0ec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe0ec0c8900 0x7fe0ec0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec06bbf0 con 0x7fe0ec15ced0 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec06b020 con 0x7fe0ec0c8900 2023-10-07T23:23:04.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1b84700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec06b2c0 con 0x7fe0ec161a70 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe0ec15ced0 0x7fe0ec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fe0ec15ced0 0x7fe0ec06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33676/0 (socket says 172.21.15.57:33676) 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 -- 172.21.15.57:0/4294683809 learned_addr learned my addr 172.21.15.57:0/4294683809 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0ebfff700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f1383700 1 --2- 172.21.15.57:0/4294683809 >> v2:172.21.15.154:3300/0 conn(0x7fe0ec0c8900 0x7fe0ec0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 -- 172.21.15.57:0/4294683809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 msgr2=0x7fe0ec15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 -- 172.21.15.57:0/4294683809 >> v2:172.21.15.154:3300/0 conn(0x7fe0ec0c8900 msgr2=0x7fe0ec0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:04.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/4294683809 >> v2:172.21.15.154:3300/0 conn(0x7fe0ec0c8900 0x7fe0ec0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 -- 172.21.15.57:0/4294683809 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec06b5a0 con 0x7fe0ec15ced0 2023-10-07T23:23:04.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.941+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec15ced0 0x7fe0ec06c080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec06a240 tx=0x7fe0dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8b1b06cd0ddad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0eb7fe700 1 -- 172.21.15.57:0/4294683809 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00b860 con 0x7fe0ec15ced0 2023-10-07T23:23:04.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0eb7fe700 1 -- 172.21.15.57:0/4294683809 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc00b9c0 con 0x7fe0ec15ced0 2023-10-07T23:23:04.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0eb7fe700 1 -- 172.21.15.57:0/4294683809 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00bd60 con 0x7fe0ec15ced0 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- 172.21.15.57:0/4294683809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec15ced0 msgr2=0x7fe0ec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec15ced0 0x7fe0ec06c080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec06a240 tx=0x7fe0dc009250 comp rx=0 tx=0).stop 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- 172.21.15.57:0/4294683809 shutdown_connections 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/4294683809 >> v2:172.21.15.154:3300/0 conn(0x7fe0ec0c8900 0x7fe0ec0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/4294683809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec15ced0 0x7fe0ec06c080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- 172.21.15.57:0/4294683809 >> 172.21.15.57:0/4294683809 conn(0x7fe0ec0670e0 msgr2=0x7fe0ec0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:04.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- 172.21.15.57:0/4294683809 shutdown_connections 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- 172.21.15.57:0/4294683809 wait complete. 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 Processor -- start 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- start start 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec0c8900 0x7fe0ec0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec162800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0c6e50 con 0x7fe0ec0c6490 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec157dd0 con 0x7fe0ec0c8900 2023-10-07T23:23:04.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1b84700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec158c30 con 0x7fe0ec161a70 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33692/0 (socket says 172.21.15.57:33692) 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 -- 172.21.15.57:0/2058194669 learned_addr learned my addr 172.21.15.57:0/2058194669 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec0c8900 0x7fe0ec0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0ebfff700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec162800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 -- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 msgr2=0x7fe0ec162800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec162800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 -- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec0c8900 msgr2=0x7fe0ec0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec0c8900 0x7fe0ec0b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:04.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.945+0000 7fe0f1383700 1 -- 172.21.15.57:0/2058194669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0094f0 con 0x7fe0ec0c6490 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0f1383700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009fa0 tx=0x7fe0e4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e400ed20 con 0x7fe0ec0c6490 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e400ee80 con 0x7fe0ec0c6490 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec1598e0 con 0x7fe0ec0c6490 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4014670 con 0x7fe0ec0c6490 2023-10-07T23:23:04.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec158300 con 0x7fe0ec0c6490 2023-10-07T23:23:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec158de0 con 0x7fe0ec0c6490 2023-10-07T23:23:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe0e40149e0 con 0x7fe0ec0c6490 2023-10-07T23:23:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 0x7fe0cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.949+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe0e407cf90 con 0x7fe0ec0c6490 2023-10-07T23:23:04.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.953+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 0x7fe0cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:04.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.953+0000 7fe0f0b82700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 0x7fe0cc062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec06a240 tx=0x7fe0dc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:04.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:04.957+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe0e40acbe0 con 0x7fe0ec0c6490 2023-10-07T23:23:05.155 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.153+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0ec06b8e0 con 0x7fe0ec0c6490 2023-10-07T23:23:05.156 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.153+0000 7fe0e8ff9700 1 -- 172.21.15.57:0/2058194669 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7fe0ec158de0 con 0x7fe0ec0c6490 2023-10-07T23:23:05.156 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:05.156 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 msgr2=0x7fe0cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 0x7fe0cc062a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec06a240 tx=0x7fe0dc017000 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 msgr2=0x7fe0ec0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009fa0 tx=0x7fe0e4009900 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 shutdown_connections 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0ec0c6490 0x7fe0ec0c6910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0ec161a70 0x7fe0ec162800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe0cc060550 0x7fe0cc062a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 --2- 172.21.15.57:0/2058194669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0ec0c8900 0x7fe0ec0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 >> 172.21.15.57:0/2058194669 conn(0x7fe0ec0670e0 msgr2=0x7fe0ec15be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 shutdown_connections 2023-10-07T23:23:05.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.157+0000 7fe0f1b84700 1 -- 172.21.15.57:0/2058194669 wait complete. 2023-10-07T23:23:05.160 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:05.171 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 Processor -- start 2023-10-07T23:23:05.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 -- start start 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 0x7efcec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcec15d2a0 0x7efcec0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcec06bbf0 con 0x7efcec161a70 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcec0c9010 con 0x7efcec15d2a0 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcf0822700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec06b020 con 0x7efcec15ced0 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcea59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7efcec161a70 0x7efcec0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:53992/0 (socket says 172.21.15.57:53992) 2023-10-07T23:23:05.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcea59c700 1 -- 172.21.15.57:0/173875505 learned_addr learned my addr 172.21.15.57:0/173875505 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 --2- 172.21.15.57:0/173875505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 0x7efcec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efceb59e700 1 --2- 172.21.15.57:0/173875505 >> v2:172.21.15.154:3300/0 conn(0x7efcec15d2a0 0x7efcec0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384839277 0 0) 0x7efcec06bbf0 con 0x7efcec161a70 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003570 con 0x7efcec161a70 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174728785 0 0) 0x7efcd8003570 con 0x7efcec161a70 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 -- 172.21.15.57:0/173875505 >> v2:172.21.15.154:3300/0 conn(0x7efcec15d2a0 msgr2=0x7efcec0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 --2- 172.21.15.57:0/173875505 >> v2:172.21.15.154:3300/0 conn(0x7efcec15d2a0 0x7efcec0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 -- 172.21.15.57:0/173875505 >> v1:172.21.15.57:6789/0 conn(0x7efcec161a70 legacy=0x7efcec0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 -- 172.21.15.57:0/173875505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec06b2c0 con 0x7efcec15ced0 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efcead9d700 1 --2- 172.21.15.57:0/173875505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 0x7efcec06c080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcd40098e0 tx=0x7efcd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbdff1626743fbe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40017f0 con 0x7efcec15ced0 2023-10-07T23:23:05.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4011420 con 0x7efcec15ced0 2023-10-07T23:23:05.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.281+0000 7efce9d9b700 1 -- 172.21.15.57:0/173875505 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd40117c0 con 0x7efcec15ced0 2023-10-07T23:23:05.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- 172.21.15.57:0/173875505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 msgr2=0x7efcec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- 172.21.15.57:0/173875505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 0x7efcec06c080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efcd40098e0 tx=0x7efcd40091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- 172.21.15.57:0/173875505 shutdown_connections 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- 172.21.15.57:0/173875505 >> v2:172.21.15.154:3300/0 conn(0x7efcec15d2a0 0x7efcec0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- 172.21.15.57:0/173875505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15ced0 0x7efcec06c080 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- 172.21.15.57:0/173875505 >> 172.21.15.57:0/173875505 conn(0x7efcec0670e0 msgr2=0x7efcec0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- 172.21.15.57:0/173875505 shutdown_connections 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- 172.21.15.57:0/173875505 wait complete. 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 Processor -- start 2023-10-07T23:23:05.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- start start 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcec15ced0 0x7efcec0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 0x7efcec148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec15dfa0 con 0x7efcec161a70 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec15d970 con 0x7efcec15ced0 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcf0822700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec0aff80 con 0x7efcec15d2a0 2023-10-07T23:23:05.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33704/0 (socket says 172.21.15.57:33704) 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 -- 172.21.15.57:0/3743494228 learned_addr learned my addr 172.21.15.57:0/3743494228 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcead9d700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcec15ced0 0x7efcec0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcea59c700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 0x7efcec148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 -- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 msgr2=0x7efcec148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 0x7efcec148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 -- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcec15ced0 msgr2=0x7efcec0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcec15ced0 0x7efcec0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 -- 172.21.15.57:0/3743494228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009420 con 0x7efcec161a70 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efcea59c700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 0x7efcec148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efceb59e700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00ea50 tx=0x7efcdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.285+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00c850 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc00c9d0 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec15dc30 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cd90 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec0b1150 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec0711c0 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efcdc013680 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efce2ffd700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 0x7efccc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efcdc07ca90 con 0x7efcec161a70 2023-10-07T23:23:05.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efcead9d700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 0x7efccc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.289+0000 7efcead9d700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 0x7efccc062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcd40030c0 tx=0x7efcd4016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.301+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efcdc04d080 con 0x7efcec161a70 2023-10-07T23:23:05.464 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.461+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcec0b0620 con 0x7efcec161a70 2023-10-07T23:23:05.465 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efce2ffd700 1 -- 172.21.15.57:0/3743494228 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+15350 (secure 0 0 0) 0x7efcdc0506a0 con 0x7efcec161a70 2023-10-07T23:23:05.466 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:05.466 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:10.194488+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 msgr2=0x7efccc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 0x7efccc062a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efcd40030c0 tx=0x7efcd4016000 comp rx=0 tx=0).stop 2023-10-07T23:23:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 msgr2=0x7efcec148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00ea50 tx=0x7efcdc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 shutdown_connections 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efcec161a70 0x7efcec148d40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efcec15d2a0 0x7efcec148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efccc060550 0x7efccc062a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 --2- 172.21.15.57:0/3743494228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcec15ced0 0x7efcec0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 >> 172.21.15.57:0/3743494228 conn(0x7efcec0670e0 msgr2=0x7efcec1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 shutdown_connections 2023-10-07T23:23:05.469 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.465+0000 7efcf0822700 1 -- 172.21.15.57:0/3743494228 wait complete. 2023-10-07T23:23:05.484 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T23:23:05.484 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 Processor -- start 2023-10-07T23:23:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- start start 2023-10-07T23:23:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9258161a70 0x7f925806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f925815d2a0 0x7f92580c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925806bbf0 con 0x7f925815ced0 2023-10-07T23:23:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580c9010 con 0x7f925815d2a0 2023-10-07T23:23:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925806b020 con 0x7f9258161a70 2023-10-07T23:23:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f925815d2a0 0x7f92580c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925dcb4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9258161a70 0x7f925806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f925815d2a0 0x7f92580c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33706/0 (socket says 172.21.15.57:33706) 2023-10-07T23:23:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 -- 172.21.15.57:0/3503905249 learned_addr learned my addr 172.21.15.57:0/3503905249 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 -- 172.21.15.57:0/3503905249 >> v2:172.21.15.154:3300/0 conn(0x7f9258161a70 msgr2=0x7f925806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 --2- 172.21.15.57:0/3503905249 >> v2:172.21.15.154:3300/0 conn(0x7f9258161a70 0x7f925806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 -- 172.21.15.57:0/3503905249 >> v1:172.21.15.154:6789/0 conn(0x7f925815ced0 legacy=0x7f92580c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 -- 172.21.15.57:0/3503905249 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925806b2c0 con 0x7f925815d2a0 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925ecb6700 1 --2- 172.21.15.57:0/3503905249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815d2a0 0x7f92580c9ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f92580b1b30 tx=0x7f9250009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2673a73e4c03bca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925d4b3700 1 -- 172.21.15.57:0/3503905249 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925000e070 con 0x7f925815d2a0 2023-10-07T23:23:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925d4b3700 1 -- 172.21.15.57:0/3503905249 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925000ba20 con 0x7f925815d2a0 2023-10-07T23:23:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925d4b3700 1 -- 172.21.15.57:0/3503905249 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500092e0 con 0x7f925815d2a0 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- 172.21.15.57:0/3503905249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815d2a0 msgr2=0x7f92580c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 --2- 172.21.15.57:0/3503905249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815d2a0 0x7f92580c9ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f92580b1b30 tx=0x7f9250009840 comp rx=0 tx=0).stop 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- 172.21.15.57:0/3503905249 shutdown_connections 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 --2- 172.21.15.57:0/3503905249 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815d2a0 0x7f92580c9ed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 --2- 172.21.15.57:0/3503905249 >> v2:172.21.15.154:3300/0 conn(0x7f9258161a70 0x7f925806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- 172.21.15.57:0/3503905249 >> 172.21.15.57:0/3503905249 conn(0x7f92580670e0 msgr2=0x7f92580674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- 172.21.15.57:0/3503905249 shutdown_connections 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.593+0000 7f925f4b7700 1 -- 172.21.15.57:0/3503905249 wait complete. 2023-10-07T23:23:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 Processor -- start 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- start start 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f925815d2a0 0x7f92580b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9258161a70 0x7f925814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580b17f0 con 0x7f925815ced0 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925815eac0 con 0x7f9258161a70 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9258150350 con 0x7f925815d2a0 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33714/0 (socket says 172.21.15.57:33714) 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 -- 172.21.15.57:0/299473287 learned_addr learned my addr 172.21.15.57:0/299473287 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925dcb4700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f925815d2a0 0x7f92580b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925ecb6700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9258161a70 0x7f925814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 -- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f925815d2a0 msgr2=0x7f92580b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f925815d2a0 0x7f92580b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 -- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9258161a70 msgr2=0x7f925814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9258161a70 0x7f925814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 -- 172.21.15.57:0/299473287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9250009ae0 con 0x7f925815ced0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925e4b5700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f924800c510 tx=0x7f924800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248011590 con 0x7f925815ced0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9248011710 con 0x7f925815ced0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925815deb0 con 0x7f925815ced0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248011ad0 con 0x7f925815ced0 2023-10-07T23:23:05.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.597+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925815e470 con 0x7f925815ced0 2023-10-07T23:23:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9258160670 con 0x7f925815ced0 2023-10-07T23:23:05.605 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f924801b3b0 con 0x7f925815ced0 2023-10-07T23:23:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f924e7fc700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 0x7f923c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f924807bbb0 con 0x7f925815ced0 2023-10-07T23:23:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f925dcb4700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 0x7f923c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.601+0000 7f925dcb4700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 0x7f923c062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92580c5690 tx=0x7f92540091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.609 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.609+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f924804c100 con 0x7f925815ced0 2023-10-07T23:23:05.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.801+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92580711c0 con 0x7f925815ced0 2023-10-07T23:23:05.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.801+0000 7f924e7fc700 1 -- 172.21.15.57:0/299473287 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f924804f720 con 0x7f925815ced0 2023-10-07T23:23:05.805 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:05.805 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 msgr2=0x7f923c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 0x7f923c062a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92580c5690 tx=0x7f92540091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 msgr2=0x7f925815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f924800c510 tx=0x7f924800c820 comp rx=0 tx=0).stop 2023-10-07T23:23:05.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 shutdown_connections 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f923c060550 0x7f923c062a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9258161a70 0x7f925814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f925815d2a0 0x7f92580b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 --2- 172.21.15.57:0/299473287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f925815ced0 0x7f925815d7d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 >> 172.21.15.57:0/299473287 conn(0x7f92580670e0 msgr2=0x7f92580cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 shutdown_connections 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.805+0000 7f925f4b7700 1 -- 172.21.15.57:0/299473287 wait complete. 2023-10-07T23:23:05.808 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:05.820 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:05.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 Processor -- start 2023-10-07T23:23:05.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 -- start start 2023-10-07T23:23:05.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ab806b850 0x7f0ab80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab806b3c0 con 0x7f0ab8158fb0 2023-10-07T23:23:05.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab806a7b0 con 0x7f0ab806b850 2023-10-07T23:23:05.925 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab7fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab806aa90 con 0x7f0ab8158be0 2023-10-07T23:23:05.926 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ab806b850 0x7f0ab80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab67fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0ab8158fb0 0x7f0ab80cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:51902/0 (socket says 172.21.15.57:51902) 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.921+0000 7f0ab67fc700 1 -- 172.21.15.57:0/1302995821 learned_addr learned my addr 172.21.15.57:0/1302995821 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab5ffb700 1 -- 172.21.15.57:0/1302995821 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483709111 0 0) 0x7f0ab806b3c0 con 0x7f0ab8158fb0 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab5ffb700 1 -- 172.21.15.57:0/1302995821 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003570 con 0x7f0ab8158fb0 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 msgr2=0x7f0ab80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab77fe700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab80ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1302995821 >> v1:172.21.15.154:6789/0 conn(0x7f0ab8158fb0 legacy=0x7f0ab80cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1302995821 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab806ad70 con 0x7f0ab806b850 2023-10-07T23:23:05.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab6ffd700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:05.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab77fe700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab80d0330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009e60 tx=0x7f0ab0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cdbceebf26e4c20 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab5ffb700 1 -- 172.21.15.57:0/1302995821 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000e070 con 0x7f0ab806b850 2023-10-07T23:23:05.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab5ffb700 1 -- 172.21.15.57:0/1302995821 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000bab0 con 0x7f0ab806b850 2023-10-07T23:23:05.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab5ffb700 1 -- 172.21.15.57:0/1302995821 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000be70 con 0x7f0ab806b850 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 msgr2=0x7f0ab80d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab80d0330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009e60 tx=0x7f0ab0009780 comp rx=0 tx=0).stop 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1302995821 shutdown_connections 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab80d0330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1302995821 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1302995821 >> 172.21.15.57:0/1302995821 conn(0x7f0ab8068520 msgr2=0x7f0ab8068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1302995821 shutdown_connections 2023-10-07T23:23:05.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.925+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1302995821 wait complete. 2023-10-07T23:23:05.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.929+0000 7f0ab7fff700 1 Processor -- start 2023-10-07T23:23:05.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.929+0000 7f0ab7fff700 1 -- start start 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab814da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab8153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8152680 con 0x7f0ab8158fb0 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80c98b0 con 0x7f0ab806b850 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8154520 con 0x7f0ab8158be0 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33724/0 (socket says 172.21.15.57:33724) 2023-10-07T23:23:05.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1742483285 learned_addr learned my addr 172.21.15.57:0/1742483285 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:05.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab6ffd700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab814da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab67fc700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab8153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.935 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 msgr2=0x7f0ab8153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab8153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 msgr2=0x7f0ab814da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab814da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 -- 172.21.15.57:0/1742483285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009420 con 0x7f0ab8158fb0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab77fe700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806ec20 tx=0x7f0ab0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0012e10 con 0x7f0ab8158fb0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0027470 con 0x7f0ab8158fb0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab0009a20 con 0x7f0ab8158fb0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0027830 con 0x7f0ab8158fb0 2023-10-07T23:23:05.936 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80c9260 con 0x7f0ab8158fb0 2023-10-07T23:23:05.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.933+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80c9a40 con 0x7f0ab8158fb0 2023-10-07T23:23:05.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.937+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0ab0024070 con 0x7f0ab8158fb0 2023-10-07T23:23:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.937+0000 7f0a9effd700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 0x7f0aa4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.937+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ab007f560 con 0x7f0ab8158fb0 2023-10-07T23:23:05.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.937+0000 7f0ab6ffd700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 0x7f0aa4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:05.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.937+0000 7f0ab6ffd700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 0x7f0aa4062a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009f40 tx=0x7f0aa80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:05.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:05.945+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0ab004fb30 con 0x7f0ab8158fb0 2023-10-07T23:23:06.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab80c9090 con 0x7f0ab8158fb0 2023-10-07T23:23:06.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0a9effd700 1 -- 172.21.15.57:0/1742483285 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4597 (secure 0 0 0) 0x7f0ab0053150 con 0x7f0ab8158fb0 2023-10-07T23:23:06.143 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:06.143 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:22:08.542997+0000","modified":"2023-10-07T23:22:12.219532+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.57:6835/2270357685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":2270357685},{"type":"v1","addr":"172.21.15.57:6835","nonce":2270357685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 msgr2=0x7f0aa4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 0x7f0aa4062a30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009f40 tx=0x7f0aa80091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 msgr2=0x7f0ab8153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ab806ec20 tx=0x7f0ab0019000 comp rx=0 tx=0).stop 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 shutdown_connections 2023-10-07T23:23:06.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab8158fb0 0x7f0ab8153fe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0ab8158be0 0x7f0ab8153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0aa4060580 0x7f0aa4062a30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 --2- 172.21.15.57:0/1742483285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab806b850 0x7f0ab814da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 >> 172.21.15.57:0/1742483285 conn(0x7f0ab8068520 msgr2=0x7f0ab8150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 shutdown_connections 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.141+0000 7f0ab7fff700 1 -- 172.21.15.57:0/1742483285 wait complete. 2023-10-07T23:23:06.146 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2023-10-07T23:23:06.157 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 Processor -- start 2023-10-07T23:23:06.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 -- start start 2023-10-07T23:23:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f4157770 0x7fb1f40af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1f4157b40 0x7fb1f415b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1f415b5a0 0x7fb1f40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f415abd0 con 0x7fb1f415b5a0 2023-10-07T23:23:06.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f415a8a0 con 0x7fb1f4157b40 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f8835700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40adef0 con 0x7fb1f4157770 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.261+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1f415b5a0 0x7fb1f40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb1f415b5a0 0x7fb1f40c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33740/0 (socket says 172.21.15.57:33740) 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 -- 172.21.15.57:0/1259636866 learned_addr learned my addr 172.21.15.57:0/1259636866 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1ebfff700 1 --2- 172.21.15.57:0/1259636866 >> v2:172.21.15.154:3300/0 conn(0x7fb1f4157b40 0x7fb1f415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f37fe700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f4157770 0x7fb1f40af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 -- 172.21.15.57:0/1259636866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f4157770 msgr2=0x7fb1f40af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f4157770 0x7fb1f40af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 -- 172.21.15.57:0/1259636866 >> v2:172.21.15.154:3300/0 conn(0x7fb1f4157b40 msgr2=0x7fb1f415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 --2- 172.21.15.57:0/1259636866 >> v2:172.21.15.154:3300/0 conn(0x7fb1f4157b40 0x7fb1f415b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 -- 172.21.15.57:0/1259636866 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40ae1d0 con 0x7fb1f415b5a0 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f3fff700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f415b5a0 0x7fb1f40c93c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406f580 tx=0x7fb1ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f720e9e349a910 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f2ffd700 1 -- 172.21.15.57:0/1259636866 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec00e070 con 0x7fb1f415b5a0 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f2ffd700 1 -- 172.21.15.57:0/1259636866 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec00b970 con 0x7fb1f415b5a0 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f2ffd700 1 -- 172.21.15.57:0/1259636866 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec0092e0 con 0x7fb1f415b5a0 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- 172.21.15.57:0/1259636866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f415b5a0 msgr2=0x7fb1f40c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f415b5a0 0x7fb1f40c93c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406f580 tx=0x7fb1ec009840 comp rx=0 tx=0).stop 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- 172.21.15.57:0/1259636866 shutdown_connections 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f415b5a0 0x7fb1f40c93c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1259636866 >> v2:172.21.15.154:3300/0 conn(0x7fb1f4157b40 0x7fb1f415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1259636866 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f4157770 0x7fb1f40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- 172.21.15.57:0/1259636866 >> 172.21.15.57:0/1259636866 conn(0x7fb1f4067b80 msgr2=0x7fb1f40c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- 172.21.15.57:0/1259636866 shutdown_connections 2023-10-07T23:23:06.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- 172.21.15.57:0/1259636866 wait complete. 2023-10-07T23:23:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 Processor -- start 2023-10-07T23:23:06.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.265+0000 7fb1f8835700 1 -- start start 2023-10-07T23:23:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f4157770 0x7fb1f4169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f415b5a0 0x7fb1f414fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4153e40 con 0x7fb1f4157b40 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40a98b0 con 0x7fb1f4157770 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4169a70 con 0x7fb1f415b5a0 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33750/0 (socket says 172.21.15.57:33750) 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 -- 172.21.15.57:0/1012455340 learned_addr learned my addr 172.21.15.57:0/1012455340 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f37fe700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f4157770 0x7fb1f4169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f3fff700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f415b5a0 0x7fb1f414fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 -- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f415b5a0 msgr2=0x7fb1f414fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f415b5a0 0x7fb1f414fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 -- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f4157770 msgr2=0x7fb1f4169530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f4157770 0x7fb1f4169530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 -- 172.21.15.57:0/1012455340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec009ae0 con 0x7fb1f4157b40 2023-10-07T23:23:06.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1ebfff700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e960 tx=0x7fb1dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc013680 con 0x7fb1f4157b40 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0137e0 con 0x7fb1f4157b40 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f41520b0 con 0x7fb1f4157b40 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc013b80 con 0x7fb1f4157b40 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.269+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f4151bf0 con 0x7fb1f4157b40 2023-10-07T23:23:06.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4153c10 con 0x7fb1f4157b40 2023-10-07T23:23:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb1dc0123d0 con 0x7fb1f4157b40 2023-10-07T23:23:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1eb7fe700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 0x7fb1d4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1dc07c8a0 con 0x7fb1f4157b40 2023-10-07T23:23:06.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1f37fe700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 0x7fb1d4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.273+0000 7fb1f37fe700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 0x7fb1d4062a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009d10 tx=0x7fb1e40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.281+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb1dc04cdf0 con 0x7fb1f4157b40 2023-10-07T23:23:06.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.445+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1f40711c0 con 0x7fb1f4157b40 2023-10-07T23:23:06.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1eb7fe700 1 -- 172.21.15.57:0/1012455340 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fb1dc017090 con 0x7fb1f4157b40 2023-10-07T23:23:06.450 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:06.451 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":35,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:22:10.194488+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:22:06.862334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-07T23:22:07.528301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 msgr2=0x7fb1d4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 0x7fb1d4062a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009d10 tx=0x7fb1e40091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 msgr2=0x7fb1f416c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00e960 tx=0x7fb1dc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.453 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 shutdown_connections 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb1f415b5a0 0x7fb1f414fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb1f4157b40 0x7fb1f416c0f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb1d4060550 0x7fb1d4062a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 --2- 172.21.15.57:0/1012455340 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f4157770 0x7fb1f4169530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 >> 172.21.15.57:0/1012455340 conn(0x7fb1f4067b80 msgr2=0x7fb1f4150bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 shutdown_connections 2023-10-07T23:23:06.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.449+0000 7fb1f8835700 1 -- 172.21.15.57:0/1012455340 wait complete. 2023-10-07T23:23:06.465 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T23:23:06.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 Processor -- start 2023-10-07T23:23:06.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 -- start start 2023-10-07T23:23:06.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa07406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07406acc0 con 0x7fa07406cd40 2023-10-07T23:23:06.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740c80f0 con 0x7fa0740cadd0 2023-10-07T23:23:06.580 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa078d94700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07406d1e0 con 0x7fa074156440 2023-10-07T23:23:06.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa073fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa07406cd40 0x7fa07406b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59868/0 (socket says 172.21.15.57:59868) 2023-10-07T23:23:06.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa073fff700 1 -- 172.21.15.57:0/1956527852 learned_addr learned my addr 172.21.15.57:0/1956527852 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:06.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa072ffd700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa07406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944843950 0 0) 0x7fa07406acc0 con 0x7fa07406cd40 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa05c003570 con 0x7fa07406cd40 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992946110 0 0) 0x7fa05c003570 con 0x7fa07406cd40 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 msgr2=0x7fa07406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa07406c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 >> v1:172.21.15.154:6789/0 conn(0x7fa0740cadd0 legacy=0x7fa0740c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07406a390 con 0x7fa07406cd40 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.577+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7fa06c0017f0 con 0x7fa07406cd40 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 >> v1:172.21.15.57:6789/0 conn(0x7fa07406cd40 legacy=0x7fa07406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa05c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c004010 0x7fa05c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c004d50 0x7fa05c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07406d1e0 con 0x7fa05c004d50 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c005c10 con 0x7fa05c004010 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c005dd0 con 0x7fa074156440 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c004d50 0x7fa05c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa072ffd700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c004010 0x7fa05c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0737fe700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa05c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 -- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 msgr2=0x7fa05c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa074156440 0x7fa05c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 -- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c004010 msgr2=0x7fa05c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa05c004010 0x7fa05c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 -- 172.21.15.57:0/1956527852 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c003570 con 0x7fa05c004d50 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c004d50 0x7fa05c0051c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00a900 tx=0x7fa0741557f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1103ee11f0fc8b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0070d0 con 0x7fa05c004d50 2023-10-07T23:23:06.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c007230 con 0x7fa05c004d50 2023-10-07T23:23:06.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa0727fc700 1 -- 172.21.15.57:0/1956527852 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0075f0 con 0x7fa05c004d50 2023-10-07T23:23:06.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 -- 172.21.15.57:0/1956527852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c004d50 msgr2=0x7fa05c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 --2- 172.21.15.57:0/1956527852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa05c004d50 0x7fa05c0051c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00a900 tx=0x7fa0741557f0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa073fff700 1 -- 172.21.15.57:0/1956527852 reap_dead start 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 -- 172.21.15.57:0/1956527852 shutdown_connections 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 -- 172.21.15.57:0/1956527852 >> 172.21.15.57:0/1956527852 conn(0x7fa0740670e0 msgr2=0x7fa0740674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 -- 172.21.15.57:0/1956527852 shutdown_connections 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.581+0000 7fa078d94700 1 -- 172.21.15.57:0/1956527852 wait complete. 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 Processor -- start 2023-10-07T23:23:06.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- start start 2023-10-07T23:23:06.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0740adc60 0x7fa0740c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741548f0 0x7fa074154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741552a0 con 0x7fa0740adc60 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074152320 con 0x7fa0741548f0 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741514b0 con 0x7fa0741527b0 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa0737fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0740adc60 0x7fa0740c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:60288/0 (socket says 172.21.15.57:60288) 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 -- 172.21.15.57:0/3136151524 learned_addr learned my addr 172.21.15.57:0/3136151524 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa073fff700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741548f0 0x7fa074154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 -- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741548f0 msgr2=0x7fa074154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741548f0 0x7fa074154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 -- 172.21.15.57:0/3136151524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0740adc60 msgr2=0x7fa0740c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0740adc60 0x7fa0740c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 -- 172.21.15.57:0/3136151524 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074152c00 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa072ffd700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa064011b70 tx=0x7fa0640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400f940 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06400faa0 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa074151710 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06400fe20 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.585+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074151cd0 con 0x7fa0741527b0 2023-10-07T23:23:06.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.589+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0741524d0 con 0x7fa0741527b0 2023-10-07T23:23:06.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.589+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa064014650 con 0x7fa0741527b0 2023-10-07T23:23:06.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.589+0000 7fa05b7fe700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 0x7fa060062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.589+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa064018030 con 0x7fa0741527b0 2023-10-07T23:23:06.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.589+0000 7fa0737fe700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 0x7fa060062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.593+0000 7fa0737fe700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 0x7fa060062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa068003a00 tx=0x7fa06800c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.597+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa06404db90 con 0x7fa0741527b0 2023-10-07T23:23:06.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.797+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa074152490 con 0x7fa0741527b0 2023-10-07T23:23:06.801 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-10-07T23:23:06.797+0000 7f9fa093f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:23:06.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.813+0000 7fa05b7fe700 1 -- 172.21.15.57:0/3136151524 <== mon.2 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa0640511b0 con 0x7fa0741527b0 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 msgr2=0x7fa060062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 0x7fa060062a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa068003a00 tx=0x7fa06800c040 comp rx=0 tx=0).stop 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 msgr2=0x7fa074150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa064011b70 tx=0x7fa0640091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 shutdown_connections 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa060060550 0x7fa060062a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0741548f0 0x7fa074154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa0741527b0 0x7fa074150f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 --2- 172.21.15.57:0/3136151524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa0740adc60 0x7fa0740c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 >> 172.21.15.57:0/3136151524 conn(0x7fa0740670e0 msgr2=0x7fa07406b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.817+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 shutdown_connections 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.821+0000 7fa078d94700 1 -- 172.21.15.57:0/3136151524 wait complete. 2023-10-07T23:23:06.822 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T23:23:06.836 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T23:23:06.852 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-10-07T23:23:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.937+0000 7f3206ce9700 1 Processor -- start 2023-10-07T23:23:06.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.937+0000 7f3206ce9700 1 -- start start 2023-10-07T23:23:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3206ce9700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 0x7f32000ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3206ce9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f32000681b0 0x7f3200068610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3206ce9700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3200159010 con 0x7f32000cb110 2023-10-07T23:23:06.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3206ce9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3200155780 con 0x7f32000681b0 2023-10-07T23:23:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3206ce9700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3200068b50 con 0x7f3200069130 2023-10-07T23:23:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f32000681b0 0x7f3200068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32054e6700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f32000cb110 0x7f3200159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59872/0 (socket says 172.21.15.57:59872) 2023-10-07T23:23:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32054e6700 1 -- 172.21.15.57:0/2568810393 learned_addr learned my addr 172.21.15.57:0/2568810393 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:06.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3205ce7700 1 --2- 172.21.15.57:0/2568810393 >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 0x7f32000ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 -- 172.21.15.57:0/2568810393 >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 msgr2=0x7f32000ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 --2- 172.21.15.57:0/2568810393 >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 0x7f32000ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3204ce5700 1 -- 172.21.15.57:0/2568810393 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664256262 0 0) 0x7f3200159010 con 0x7f32000cb110 2023-10-07T23:23:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 -- 172.21.15.57:0/2568810393 >> v1:172.21.15.57:6789/0 conn(0x7f32000cb110 legacy=0x7f3200159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.947 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 -- 172.21.15.57:0/2568810393 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000cb880 con 0x7f32000681b0 2023-10-07T23:23:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3205ce7700 1 --2- 172.21.15.57:0/2568810393 >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 0x7f32000ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f32064e8700 1 --2- 172.21.15.57:0/2568810393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f3200068610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f320006f5a0 tx=0x7f31f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc910e235c53557 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3204ce5700 1 -- 172.21.15.57:0/2568810393 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800e070 con 0x7f32000681b0 2023-10-07T23:23:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3204ce5700 1 -- 172.21.15.57:0/2568810393 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f800bbf0 con 0x7f32000681b0 2023-10-07T23:23:06.948 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.941+0000 7f3204ce5700 1 -- 172.21.15.57:0/2568810393 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8012430 con 0x7f32000681b0 2023-10-07T23:23:06.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 -- 172.21.15.57:0/2568810393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 msgr2=0x7f3200068610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 --2- 172.21.15.57:0/2568810393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f3200068610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f320006f5a0 tx=0x7f31f8009780 comp rx=0 tx=0).stop 2023-10-07T23:23:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 -- 172.21.15.57:0/2568810393 shutdown_connections 2023-10-07T23:23:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 --2- 172.21.15.57:0/2568810393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f3200068610 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 --2- 172.21.15.57:0/2568810393 >> v2:172.21.15.154:3300/0 conn(0x7f3200069130 0x7f32000ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 -- 172.21.15.57:0/2568810393 >> 172.21.15.57:0/2568810393 conn(0x7f32000c9740 msgr2=0x7f320006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:06.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 -- 172.21.15.57:0/2568810393 shutdown_connections 2023-10-07T23:23:06.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 -- 172.21.15.57:0/2568810393 wait complete. 2023-10-07T23:23:06.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.945+0000 7f3206ce9700 1 Processor -- start 2023-10-07T23:23:06.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- start start 2023-10-07T23:23:06.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3200069130 0x7f32001562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 0x7f320013ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200158140 con 0x7f32000681b0 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320013b170 con 0x7f3200069130 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32000c4a40 con 0x7f32000cb110 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38918/0 (socket says 172.21.15.57:38918) 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 -- 172.21.15.57:0/176609958 learned_addr learned my addr 172.21.15.57:0/176609958 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f32064e8700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 0x7f320013ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f32054e6700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3200069130 0x7f32001562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 -- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 msgr2=0x7f320013ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 0x7f320013ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 -- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3200069130 msgr2=0x7f32001562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3200069130 0x7f32001562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 -- 172.21.15.57:0/176609958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009a20 con 0x7f32000681b0 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f32064e8700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 0x7f320013ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3205ce7700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00d860 tx=0x7f31fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0125f0 con 0x7f32000681b0 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc012750 con 0x7f32000681b0 2023-10-07T23:23:06.955 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32000c4d20 con 0x7f32000681b0 2023-10-07T23:23:06.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc012af0 con 0x7f32000681b0 2023-10-07T23:23:06.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.949+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3200157dd0 con 0x7f32000681b0 2023-10-07T23:23:06.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32000711c0 con 0x7f32000681b0 2023-10-07T23:23:06.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f31fc013480 con 0x7f32000681b0 2023-10-07T23:23:06.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f31f5ffb700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 0x7f31e4062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:06.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f31fc07c8c0 con 0x7f32000681b0 2023-10-07T23:23:06.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f32054e6700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 0x7f31e4062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:06.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.953+0000 7f32054e6700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 0x7f31e4062a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f320013b870 tx=0x7f31f0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:06.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:06.965+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f31fc04ce60 con 0x7f32000681b0 2023-10-07T23:23:07.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.157+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f32000c4f10 con 0x7f32000681b0 2023-10-07T23:23:07.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.813+0000 7f31f5ffb700 1 -- 172.21.15.57:0/176609958 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f31fc050480 con 0x7f32000681b0 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 msgr2=0x7f31e4062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 0x7f31e4062a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f320013b870 tx=0x7f31f0007d30 comp rx=0 tx=0).stop 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 msgr2=0x7f32001585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00d860 tx=0x7f31fc008040 comp rx=0 tx=0).stop 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 shutdown_connections 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32000cb110 0x7f320013ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f31e4060580 0x7f31e4062a30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3200069130 0x7f32001562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 --2- 172.21.15.57:0/176609958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32000681b0 0x7f32001585d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 >> 172.21.15.57:0/176609958 conn(0x7f32000c9740 msgr2=0x7f32000c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 shutdown_connections 2023-10-07T23:23:07.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.817+0000 7f3206ce9700 1 -- 172.21.15.57:0/176609958 wait complete. 2023-10-07T23:23:07.839 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T23:23:07.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 Processor -- start 2023-10-07T23:23:07.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 -- start start 2023-10-07T23:23:07.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d50159010 con 0x7f3d500cb110 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d50068b70 con 0x7f3d500681b0 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d54a92700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50155780 con 0x7f3d50069130 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d4ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d4f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d46ffd700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3d500cb110 0x7f3d50159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:58772/0 (socket says 172.21.15.57:58772) 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d46ffd700 1 -- 172.21.15.57:0/2241648592 learned_addr learned my addr 172.21.15.57:0/2241648592 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2241648592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:60322/0 (socket says 172.21.15.57:60322) 2023-10-07T23:23:07.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.961+0000 7f3d4ffff700 1 --2- 172.21.15.57:0/2241648592 >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36892/0 (socket says 172.21.15.57:36892) 2023-10-07T23:23:07.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4f7fe700 1 -- 172.21.15.57:0/2241648592 >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 msgr2=0x7f3d50068630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2241648592 >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4f7fe700 1 -- 172.21.15.57:0/2241648592 >> v1:172.21.15.154:6789/0 conn(0x7f3d500cb110 legacy=0x7f3d50159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4f7fe700 1 -- 172.21.15.57:0/2241648592 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500cb880 con 0x7f3d50069130 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4effd700 1 -- 172.21.15.57:0/2241648592 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875984215 0 0) 0x7f3d50159010 con 0x7f3d500cb110 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4ffff700 1 --2- 172.21.15.57:0/2241648592 >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2241648592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d400098e0 tx=0x7f3d400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9750dc7c1f9bd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4effd700 1 -- 172.21.15.57:0/2241648592 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000b640 con 0x7f3d50069130 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4effd700 1 -- 172.21.15.57:0/2241648592 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4000b7a0 con 0x7f3d50069130 2023-10-07T23:23:07.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d4effd700 1 -- 172.21.15.57:0/2241648592 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000bb40 con 0x7f3d50069130 2023-10-07T23:23:07.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 -- 172.21.15.57:0/2241648592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 msgr2=0x7f3d500ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2241648592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d400098e0 tx=0x7f3d400091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 -- 172.21.15.57:0/2241648592 shutdown_connections 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2241648592 >> v2:172.21.15.154:3300/0 conn(0x7f3d500681b0 0x7f3d50068630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2241648592 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d50069130 0x7f3d500ca8d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 -- 172.21.15.57:0/2241648592 >> 172.21.15.57:0/2241648592 conn(0x7f3d500c9740 msgr2=0x7f3d5006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 -- 172.21.15.57:0/2241648592 shutdown_connections 2023-10-07T23:23:07.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.965+0000 7f3d54a92700 1 -- 172.21.15.57:0/2241648592 wait complete. 2023-10-07T23:23:07.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.969+0000 7f3d54a92700 1 Processor -- start 2023-10-07T23:23:07.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.969+0000 7f3d54a92700 1 -- start start 2023-10-07T23:23:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.969+0000 7f3d54a92700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d500681b0 0x7f3d50152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.969+0000 7f3d54a92700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.969+0000 7f3d54a92700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d500cb110 0x7f3d500b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50153270 con 0x7f3d50069130 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500ad000 con 0x7f3d500cb110 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d50156380 con 0x7f3d500681b0 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38922/0 (socket says 172.21.15.57:38922) 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 -- 172.21.15.57:0/2207029232 learned_addr learned my addr 172.21.15.57:0/2207029232 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d500681b0 0x7f3d50152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d4ffff700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d500cb110 0x7f3d500b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 -- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d500681b0 msgr2=0x7f3d50152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d500681b0 0x7f3d50152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 -- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d500cb110 msgr2=0x7f3d500b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d500cb110 0x7f3d500b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:07.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 -- 172.21.15.57:0/2207029232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4800ab30 con 0x7f3d50069130 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d46ffd700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800c810 tx=0x7f3d38007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38005b30 con 0x7f3d50069130 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38005c90 con 0x7f3d50069130 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d40009420 con 0x7f3d50069130 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380072e0 con 0x7f3d50069130 2023-10-07T23:23:07.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d501568d0 con 0x7f3d50069130 2023-10-07T23:23:07.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3d380076d0 con 0x7f3d50069130 2023-10-07T23:23:07.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.973+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d500711c0 con 0x7f3d50069130 2023-10-07T23:23:07.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.977+0000 7f3d47fff700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 0x7f3d30062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:07.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.977+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3d3807bd10 con 0x7f3d50069130 2023-10-07T23:23:07.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.977+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 0x7f3d30062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:07.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.977+0000 7f3d4f7fe700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 0x7f3d30062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d400093f0 tx=0x7f3d40017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:07.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:07.985+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3d3804c360 con 0x7f3d50069130 2023-10-07T23:23:08.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:08.149+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d500ab630 con 0x7f3d50069130 2023-10-07T23:23:09.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.021+0000 7f3d47fff700 1 -- 172.21.15.57:0/2207029232 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3d3804f980 con 0x7f3d50069130 2023-10-07T23:23:09.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 msgr2=0x7f3d30062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:09.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 0x7f3d30062a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d400093f0 tx=0x7f3d40017000 comp rx=0 tx=0).stop 2023-10-07T23:23:09.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 msgr2=0x7f3d500b0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:09.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800c810 tx=0x7f3d38007df0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 shutdown_connections 2023-10-07T23:23:09.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d30060550 0x7f3d30062a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d500cb110 0x7f3d500b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d50069130 0x7f3d500b0230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 --2- 172.21.15.57:0/2207029232 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d500681b0 0x7f3d50152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 >> 172.21.15.57:0/2207029232 conn(0x7f3d500c9740 msgr2=0x7f3d500c2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 shutdown_connections 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.029+0000 7f3d54a92700 1 -- 172.21.15.57:0/2207029232 wait complete. 2023-10-07T23:23:09.031 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-10-07T23:23:09.047 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T23:23:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.153+0000 7f7186069700 1 Processor -- start 2023-10-07T23:23:09.157 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.153+0000 7f7186069700 1 -- start start 2023-10-07T23:23:09.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7180158c40 0x7f7180159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 0x7f718006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71800afd20 0x7f718006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718015d9f0 con 0x7f7180158c40 2023-10-07T23:23:09.159 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718006ac30 con 0x7f71800afd20 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800ca530 con 0x7f718015d580 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7180158c40 0x7f7180159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7180158c40 0x7f7180159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38928/0 (socket says 172.21.15.57:38928) 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 -- 172.21.15.57:0/2874357045 learned_addr learned my addr 172.21.15.57:0/2874357045 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185868700 1 --2- 172.21.15.57:0/2874357045 >> v2:172.21.15.154:3300/0 conn(0x7f71800afd20 0x7f718006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7184866700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 0x7f718006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 -- 172.21.15.57:0/2874357045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 msgr2=0x7f718006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 0x7f718006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 -- 172.21.15.57:0/2874357045 >> v2:172.21.15.154:3300/0 conn(0x7f71800afd20 msgr2=0x7f718006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 --2- 172.21.15.57:0/2874357045 >> v2:172.21.15.154:3300/0 conn(0x7f71800afd20 0x7f718006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 -- 172.21.15.57:0/2874357045 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718006ae60 con 0x7f7180158c40 2023-10-07T23:23:09.160 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7184866700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 0x7f718006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:23:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7185067700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7180158c40 0x7f7180159010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f718006a020 tx=0x7f717c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec11d6a75e3d7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7177fff700 1 -- 172.21.15.57:0/2874357045 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00b860 con 0x7f7180158c40 2023-10-07T23:23:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7177fff700 1 -- 172.21.15.57:0/2874357045 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00b9c0 con 0x7f7180158c40 2023-10-07T23:23:09.161 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7177fff700 1 -- 172.21.15.57:0/2874357045 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00bd60 con 0x7f7180158c40 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- 172.21.15.57:0/2874357045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7180158c40 msgr2=0x7f7180159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7180158c40 0x7f7180159010 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f718006a020 tx=0x7f717c009250 comp rx=0 tx=0).stop 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- 172.21.15.57:0/2874357045 shutdown_connections 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- 172.21.15.57:0/2874357045 >> v2:172.21.15.154:3300/0 conn(0x7f71800afd20 0x7f718006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f718015d580 0x7f718006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 --2- 172.21.15.57:0/2874357045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7180158c40 0x7f7180159010 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- 172.21.15.57:0/2874357045 >> 172.21.15.57:0/2874357045 conn(0x7f7180062ba0 msgr2=0x7f7180062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- 172.21.15.57:0/2874357045 shutdown_connections 2023-10-07T23:23:09.162 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.157+0000 7f7186069700 1 -- 172.21.15.57:0/2874357045 wait complete. 2023-10-07T23:23:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 Processor -- start 2023-10-07T23:23:09.163 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 -- start start 2023-10-07T23:23:09.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71800afd20 0x7f7180155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 0x7f71800c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180154160 con 0x7f718015d580 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180154400 con 0x7f7180158c40 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.161+0000 7f7186069700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71801546e0 con 0x7f71800afd20 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38936/0 (socket says 172.21.15.57:38936) 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185067700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71800afd20 0x7f7180155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 -- 172.21.15.57:0/3741163151 learned_addr learned my addr 172.21.15.57:0/3741163151 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7184866700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 0x7f71800c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 -- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71800afd20 msgr2=0x7f7180155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71800afd20 0x7f7180155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 -- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 msgr2=0x7f71800c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 0x7f71800c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 -- 172.21.15.57:0/3741163151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0094f0 con 0x7f718015d580 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7184866700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 0x7f71800c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7185868700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7178009fa0 tx=0x7f7178009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717800ec70 con 0x7f718015d580 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71801549c0 con 0x7f718015d580 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717800edd0 con 0x7f718015d580 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178014670 con 0x7f718015d580 2023-10-07T23:23:09.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71801561a0 con 0x7f718015d580 2023-10-07T23:23:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.165+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7180154ec0 con 0x7f718015d580 2023-10-07T23:23:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.169+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f71780149e0 con 0x7f718015d580 2023-10-07T23:23:09.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.169+0000 7f71757fa700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 0x7f7164062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:09.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.169+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f717807cdc0 con 0x7f718015d580 2023-10-07T23:23:09.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.169+0000 7f7185067700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 0x7f7164062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:09.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.169+0000 7f7185067700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 0x7f7164062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f718006a020 tx=0x7f717c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:09.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.177+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f717804d510 con 0x7f718015d580 2023-10-07T23:23:09.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:09.333+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71800711c0 con 0x7f718015d580 2023-10-07T23:23:10.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.029+0000 7f71757fa700 1 -- 172.21.15.57:0/3741163151 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7178050b30 con 0x7f718015d580 2023-10-07T23:23:10.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 msgr2=0x7f7164062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 0x7f7164062a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f718006a020 tx=0x7f717c017000 comp rx=0 tx=0).stop 2023-10-07T23:23:10.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 msgr2=0x7f71800b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7178009fa0 tx=0x7f7178009900 comp rx=0 tx=0).stop 2023-10-07T23:23:10.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 shutdown_connections 2023-10-07T23:23:10.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f718015d580 0x7f71800b13c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7164060550 0x7f7164062a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7180158c40 0x7f71800c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 --2- 172.21.15.57:0/3741163151 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f71800afd20 0x7f7180155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 >> 172.21.15.57:0/3741163151 conn(0x7f7180062ba0 msgr2=0x7f71800670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 shutdown_connections 2023-10-07T23:23:10.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.033+0000 7f7186069700 1 -- 172.21.15.57:0/3741163151 wait complete. 2023-10-07T23:23:10.038 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-10-07T23:23:10.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2023-10-07T23:23:10.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.161+0000 7f0930c7f700 1 Processor -- start 2023-10-07T23:23:10.166 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.161+0000 7f0930c7f700 1 -- start start 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0930c7f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f092c0c9a20 0x7f092c069440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0930c7f700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0abc50 con 0x7f092c1618e0 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0930c7f700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c06db20 con 0x7f092c161cb0 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0930c7f700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c0c7c00 con 0x7f092c0c9a20 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f092a59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f092c1618e0 0x7f092c0c74f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59884/0 (socket says 172.21.15.57:59884) 2023-10-07T23:23:10.167 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f092a59c700 1 -- 172.21.15.57:0/1219546989 learned_addr learned my addr 172.21.15.57:0/1219546989 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0929d9b700 1 --2- 172.21.15.57:0/1219546989 >> v2:172.21.15.57:3300/0 conn(0x7f092c0c9a20 0x7f092c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0929d9b700 1 -- 172.21.15.57:0/1219546989 >> v1:172.21.15.154:6789/0 conn(0x7f092c161cb0 legacy=0x7f092c06c800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-07T23:23:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0929d9b700 1 -- 172.21.15.57:0/1219546989 >> v1:172.21.15.57:6789/0 conn(0x7f092c1618e0 legacy=0x7f092c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0929d9b700 1 -- 172.21.15.57:0/1219546989 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c06dd90 con 0x7f092c0c9a20 2023-10-07T23:23:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.165+0000 7f0929d9b700 1 --2- 172.21.15.57:0/1219546989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c069440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f09140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2efc44d6ac52ba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f092959a700 1 -- 172.21.15.57:0/1219546989 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914009e80 con 0x7f092c0c9a20 2023-10-07T23:23:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f092959a700 1 -- 172.21.15.57:0/1219546989 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400b820 con 0x7f092c0c9a20 2023-10-07T23:23:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f092959a700 1 -- 172.21.15.57:0/1219546989 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400bbe0 con 0x7f092c0c9a20 2023-10-07T23:23:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 -- 172.21.15.57:0/1219546989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 msgr2=0x7f092c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 --2- 172.21.15.57:0/1219546989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c069440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f09140091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 -- 172.21.15.57:0/1219546989 shutdown_connections 2023-10-07T23:23:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 --2- 172.21.15.57:0/1219546989 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c069440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 -- 172.21.15.57:0/1219546989 >> 172.21.15.57:0/1219546989 conn(0x7f092c0ccf10 msgr2=0x7f092c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 -- 172.21.15.57:0/1219546989 shutdown_connections 2023-10-07T23:23:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.169+0000 7f0930c7f700 1 -- 172.21.15.57:0/1219546989 wait complete. 2023-10-07T23:23:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 Processor -- start 2023-10-07T23:23:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- start start 2023-10-07T23:23:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f092c1618e0 0x7f092c16f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f092c161cb0 0x7f092c0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c172470 con 0x7f092c0c9a20 2023-10-07T23:23:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0b1330 con 0x7f092c161cb0 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0b0a60 con 0x7f092c1618e0 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38950/0 (socket says 172.21.15.57:38950) 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 -- 172.21.15.57:0/2194074832 learned_addr learned my addr 172.21.15.57:0/2194074832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092ad9d700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f092c161cb0 0x7f092c0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0929d9b700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f092c1618e0 0x7f092c16f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 -- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f092c1618e0 msgr2=0x7f092c16f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f092c1618e0 0x7f092c16f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 -- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f092c161cb0 msgr2=0x7f092c0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f092c161cb0 0x7f092c0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 -- 172.21.15.57:0/2194074832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009420 con 0x7f092c0c9a20 2023-10-07T23:23:10.177 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f092a59c700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f092400c7c0 tx=0x7f0924007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924005b30 con 0x7f092c0c9a20 2023-10-07T23:23:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924005c90 con 0x7f092c0c9a20 2023-10-07T23:23:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c172190 con 0x7f092c0c9a20 2023-10-07T23:23:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924007300 con 0x7f092c0c9a20 2023-10-07T23:23:10.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.173+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0b0fc0 con 0x7f092c0c9a20 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c0647b0 con 0x7f092c0c9a20 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0924007670 con 0x7f092c0c9a20 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f09227fc700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 0x7f090c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f092407b8e0 con 0x7f092c0c9a20 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f0929d9b700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 0x7f090c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.177+0000 7f0929d9b700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 0x7f090c062a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f0914016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.181+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f092404c1d0 con 0x7f092c0c9a20 2023-10-07T23:23:10.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.337+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) v1 -- 0x7f092c0ab8e0 con 0x7f092c0c9a20 2023-10-07T23:23:10.347 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.345+0000 7f09227fc700 1 -- 172.21.15.57:0/2194074832 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v13) v1 ==== 119+0+0 (secure 0 0 0) 0x7f0924053e60 con 0x7f092c0c9a20 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 msgr2=0x7f090c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 0x7f090c062a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09140098e0 tx=0x7f0914016000 comp rx=0 tx=0).stop 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 msgr2=0x7f092c148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f092400c7c0 tx=0x7f0924007f20 comp rx=0 tx=0).stop 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 shutdown_connections 2023-10-07T23:23:10.350 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f090c060550 0x7f090c062a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f092c161cb0 0x7f092c0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f092c1618e0 0x7f092c16f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 --2- 172.21.15.57:0/2194074832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f092c0c9a20 0x7f092c148980 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 >> 172.21.15.57:0/2194074832 conn(0x7f092c0ccf10 msgr2=0x7f092c160510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 shutdown_connections 2023-10-07T23:23:10.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.349+0000 7f0930c7f700 1 -- 172.21.15.57:0/2194074832 wait complete. 2023-10-07T23:23:10.363 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2023-10-07T23:23:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.469+0000 7f4f19014700 1 Processor -- start 2023-10-07T23:23:10.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.469+0000 7f4f19014700 1 -- start start 2023-10-07T23:23:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.469+0000 7f4f19014700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.469+0000 7f4f19014700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1406d180 con 0x7f4f14159080 2023-10-07T23:23:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f140cb910 con 0x7f4f1406cd40 2023-10-07T23:23:10.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140ca590 con 0x7f4f14159450 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f12d9d700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4f14159080 0x7f4f1415d750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:58796/0 (socket says 172.21.15.57:58796) 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1359e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:36916/0 (socket says 172.21.15.57:36916) 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:60370/0 (socket says 172.21.15.57:60370) 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1359e700 1 -- 172.21.15.57:0/3523574247 learned_addr learned my addr 172.21.15.57:0/3523574247 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:10.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 -- 172.21.15.57:0/3523574247 >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 msgr2=0x7f4f1406a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 --2- 172.21.15.57:0/3523574247 >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 -- 172.21.15.57:0/3523574247 >> v1:172.21.15.154:6789/0 conn(0x7f4f14159080 legacy=0x7f4f1415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 -- 172.21.15.57:0/3523574247 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140ca870 con 0x7f4f14159450 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1359e700 1 --2- 172.21.15.57:0/3523574247 >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f1259c700 1 --2- 172.21.15.57:0/3523574247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098e0 tx=0x7f4efc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0980996b911cb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f11d9b700 1 -- 172.21.15.57:0/3523574247 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00b640 con 0x7f4f14159450 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f11d9b700 1 -- 172.21.15.57:0/3523574247 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc00b7a0 con 0x7f4f14159450 2023-10-07T23:23:10.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f11d9b700 1 -- 172.21.15.57:0/3523574247 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00bb40 con 0x7f4f14159450 2023-10-07T23:23:10.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- 172.21.15.57:0/3523574247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 msgr2=0x7f4f1406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 --2- 172.21.15.57:0/3523574247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098e0 tx=0x7f4efc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- 172.21.15.57:0/3523574247 shutdown_connections 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 --2- 172.21.15.57:0/3523574247 >> v2:172.21.15.154:3300/0 conn(0x7f4f1406cd40 0x7f4f1406a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 --2- 172.21.15.57:0/3523574247 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f14159450 0x7f4f1406c800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- 172.21.15.57:0/3523574247 >> 172.21.15.57:0/3523574247 conn(0x7f4f14062ba0 msgr2=0x7f4f14062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- 172.21.15.57:0/3523574247 shutdown_connections 2023-10-07T23:23:10.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.473+0000 7f4f19014700 1 -- 172.21.15.57:0/3523574247 wait complete. 2023-10-07T23:23:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 Processor -- start 2023-10-07T23:23:10.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 -- start start 2023-10-07T23:23:10.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f1406cd40 0x7f4f140c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f14159080 0x7f4f1416a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140c3850 con 0x7f4f14159450 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f140c3b30 con 0x7f4f14159080 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f19014700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1416adc0 con 0x7f4f1406cd40 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f14159080 0x7f4f1416a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f12d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f1406cd40 0x7f4f140c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38960/0 (socket says 172.21.15.57:38960) 2023-10-07T23:23:10.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 -- 172.21.15.57:0/1870239316 learned_addr learned my addr 172.21.15.57:0/1870239316 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 -- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f1406cd40 msgr2=0x7f4f140c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f1406cd40 0x7f4f140c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 -- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f14159080 msgr2=0x7f4f1416a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f14159080 0x7f4f1416a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 -- 172.21.15.57:0/1870239316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009ae0 con 0x7f4f14159450 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f1359e700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c0c0 tx=0x7f4f0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c018070 con 0x7f4f14159450 2023-10-07T23:23:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.477+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c00f040 con 0x7f4f14159450 2023-10-07T23:23:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc009420 con 0x7f4f14159450 2023-10-07T23:23:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00cc00 con 0x7f4f14159450 2023-10-07T23:23:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1416a1e0 con 0x7f4f14159450 2023-10-07T23:23:10.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f140c3ca0 con 0x7f4f14159450 2023-10-07T23:23:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4f0c003020 con 0x7f4f14159450 2023-10-07T23:23:10.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f0affd700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 0x7f4ef4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4f0c07c490 con 0x7f4f14159450 2023-10-07T23:23:10.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.481+0000 7f4f12d9d700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 0x7f4ef4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.486 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.485+0000 7f4f12d9d700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 0x7f4ef4062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007fd0 tx=0x7f4f04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.493+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4f0c04cce0 con 0x7f4f14159450 2023-10-07T23:23:10.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.649+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) v1 -- 0x7f4f140647b0 con 0x7f4f14159450 2023-10-07T23:23:10.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.661+0000 7f4f0affd700 1 -- 172.21.15.57:0/1870239316 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v14) v1 ==== 106+0+0 (secure 0 0 0) 0x7f4f0c050300 con 0x7f4f14159450 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 msgr2=0x7f4ef4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 0x7f4ef4062a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007fd0 tx=0x7f4f04009040 comp rx=0 tx=0).stop 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 msgr2=0x7f4f141696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c0c0 tx=0x7f4f0c00b040 comp rx=0 tx=0).stop 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 shutdown_connections 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4f14159450 0x7f4f141696b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4ef4060550 0x7f4ef4062a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f14159080 0x7f4f1416a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 --2- 172.21.15.57:0/1870239316 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f4f1406cd40 0x7f4f140c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 >> 172.21.15.57:0/1870239316 conn(0x7f4f14062ba0 msgr2=0x7f4f140cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 shutdown_connections 2023-10-07T23:23:10.668 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.665+0000 7f4f19014700 1 -- 172.21.15.57:0/1870239316 wait complete. 2023-10-07T23:23:10.680 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-07T23:23:10.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.793+0000 7f2e38b83700 1 Processor -- start 2023-10-07T23:23:10.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.793+0000 7f2e38b83700 1 -- start start 2023-10-07T23:23:10.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2e340c7d40 0x7f2e3406de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e340cb4e0 0x7f2e341560c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e340cb050 con 0x7f2e3406da40 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340c81d0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340c84b0 con 0x7f2e340c7d40 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e340cb4e0 0x7f2e341560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e340cb4e0 0x7f2e341560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38966/0 (socket says 172.21.15.57:38966) 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 -- 172.21.15.57:0/1857884263 learned_addr learned my addr 172.21.15.57:0/1857884263 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e2affd700 1 --2- 172.21.15.57:0/1857884263 >> v2:172.21.15.154:3300/0 conn(0x7f2e340c7d40 0x7f2e3406de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 -- 172.21.15.57:0/1857884263 >> v2:172.21.15.154:3300/0 conn(0x7f2e340c7d40 msgr2=0x7f2e3406de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 --2- 172.21.15.57:0/1857884263 >> v2:172.21.15.154:3300/0 conn(0x7f2e340c7d40 0x7f2e3406de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 -- 172.21.15.57:0/1857884263 >> v1:172.21.15.154:6789/0 conn(0x7f2e3406da40 legacy=0x7f2e3406b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 -- 172.21.15.57:0/1857884263 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340c8790 con 0x7f2e340cb4e0 2023-10-07T23:23:10.801 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e33fff700 1 --2- 172.21.15.57:0/1857884263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e341560c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009f20 tx=0x7f2e2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca65c7f716c2927f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e32ffd700 1 -- 172.21.15.57:0/1857884263 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00e070 con 0x7f2e340cb4e0 2023-10-07T23:23:10.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e32ffd700 1 -- 172.21.15.57:0/1857884263 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c00b990 con 0x7f2e340cb4e0 2023-10-07T23:23:10.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e32ffd700 1 -- 172.21.15.57:0/1857884263 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0092e0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- 172.21.15.57:0/1857884263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 msgr2=0x7f2e341560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 --2- 172.21.15.57:0/1857884263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e341560c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009f20 tx=0x7f2e2c009840 comp rx=0 tx=0).stop 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- 172.21.15.57:0/1857884263 shutdown_connections 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 --2- 172.21.15.57:0/1857884263 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e341560c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 --2- 172.21.15.57:0/1857884263 >> v2:172.21.15.154:3300/0 conn(0x7f2e340c7d40 0x7f2e3406de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.797+0000 7f2e38b83700 1 -- 172.21.15.57:0/1857884263 >> 172.21.15.57:0/1857884263 conn(0x7f2e340c9740 msgr2=0x7f2e3406acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- 172.21.15.57:0/1857884263 shutdown_connections 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- 172.21.15.57:0/1857884263 wait complete. 2023-10-07T23:23:10.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 Processor -- start 2023-10-07T23:23:10.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- start start 2023-10-07T23:23:10.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3406da40 0x7f2e340ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e340c7d40 0x7f2e34155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340c3c80 con 0x7f2e340cb4e0 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340af060 con 0x7f2e3406da40 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e38b83700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340c3e90 con 0x7f2e340c7d40 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e33fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e33fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38982/0 (socket says 172.21.15.57:38982) 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e33fff700 1 -- 172.21.15.57:0/3985642890 learned_addr learned my addr 172.21.15.57:0/3985642890 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e337fe700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3406da40 0x7f2e340ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.801+0000 7f2e2affd700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e340c7d40 0x7f2e34155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 -- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e340c7d40 msgr2=0x7f2e34155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e340c7d40 0x7f2e34155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 -- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3406da40 msgr2=0x7f2e340ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3406da40 0x7f2e340ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 -- 172.21.15.57:0/3985642890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c009ae0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.807 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e33fff700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009880 tx=0x7f2e2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c008ba0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c008d00 con 0x7f2e340cb4e0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c0256b0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e340c4170 con 0x7f2e340cb4e0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e340c4500 con 0x7f2e340cb4e0 2023-10-07T23:23:10.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e340711c0 con 0x7f2e340cb4e0 2023-10-07T23:23:10.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2e2c003040 con 0x7f2e340cb4e0 2023-10-07T23:23:10.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.805+0000 7f2e2bfff700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 0x7f2e14062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:10.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.809+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2e2c07da30 con 0x7f2e340cb4e0 2023-10-07T23:23:10.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.809+0000 7f2e337fe700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 0x7f2e14062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:10.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.809+0000 7f2e337fe700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 0x7f2e14062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e24003a80 tx=0x7f2e24007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:10.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.817+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2e2c04e320 con 0x7f2e340cb4e0 2023-10-07T23:23:10.983 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.981+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2e340c8a70 con 0x7f2e340cb4e0 2023-10-07T23:23:10.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.985+0000 7f2e2bfff700 1 -- 172.21.15.57:0/3985642890 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2e2c051940 con 0x7f2e340cb4e0 2023-10-07T23:23:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 msgr2=0x7f2e14062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 0x7f2e14062a00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e24003a80 tx=0x7f2e24007ec0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 msgr2=0x7f2e34155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009880 tx=0x7f2e2c018000 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 shutdown_connections 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e340cb4e0 0x7f2e34155ac0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e340c7d40 0x7f2e34155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e14060550 0x7f2e14062a00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 --2- 172.21.15.57:0/3985642890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e3406da40 0x7f2e340ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 >> 172.21.15.57:0/3985642890 conn(0x7f2e340c9740 msgr2=0x7f2e34152600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 shutdown_connections 2023-10-07T23:23:10.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:10.989+0000 7f2e38b83700 1 -- 172.21.15.57:0/3985642890 wait complete. 2023-10-07T23:23:11.004 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2023-10-07T23:23:11.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 Processor -- start 2023-10-07T23:23:11.114 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 -- start start 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff7f40c9ad0 0x7ff7f4156180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff7f406ac90 0x7ff7f41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f406caa0 con 0x7ff7f4155db0 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40ad090 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f9769700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f406b1e0 con 0x7ff7f406ac90 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff7f40c9ad0 0x7ff7f4156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff7f40c9ad0 0x7ff7f4156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38984/0 (socket says 172.21.15.57:38984) 2023-10-07T23:23:11.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3932853541 learned_addr learned my addr 172.21.15.57:0/3932853541 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f8f68700 1 --2- 172.21.15.57:0/3932853541 >> v2:172.21.15.154:3300/0 conn(0x7ff7f406ac90 0x7ff7f41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3932853541 >> v2:172.21.15.154:3300/0 conn(0x7ff7f406ac90 msgr2=0x7ff7f41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 --2- 172.21.15.57:0/3932853541 >> v2:172.21.15.154:3300/0 conn(0x7ff7f406ac90 0x7ff7f41566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3932853541 >> v1:172.21.15.154:6789/0 conn(0x7ff7f4155db0 legacy=0x7ff7f40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3932853541 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f406b4c0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.113+0000 7ff7f37fe700 1 --2- 172.21.15.57:0/3932853541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f4156180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406a710 tx=0x7ff7e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc280eae631a5822 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f2ffd700 1 -- 172.21.15.57:0/3932853541 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8009e70 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f2ffd700 1 -- 172.21.15.57:0/3932853541 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800b860 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f2ffd700 1 -- 172.21.15.57:0/3932853541 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800bc00 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- 172.21.15.57:0/3932853541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 msgr2=0x7ff7f4156180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3932853541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f4156180 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff7f406a710 tx=0x7ff7e8009250 comp rx=0 tx=0).stop 2023-10-07T23:23:11.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- 172.21.15.57:0/3932853541 shutdown_connections 2023-10-07T23:23:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3932853541 >> v2:172.21.15.154:3300/0 conn(0x7ff7f406ac90 0x7ff7f41566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3932853541 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f4156180 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:11.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- 172.21.15.57:0/3932853541 >> 172.21.15.57:0/3932853541 conn(0x7ff7f4067b80 msgr2=0x7ff7f40c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- 172.21.15.57:0/3932853541 shutdown_connections 2023-10-07T23:23:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- 172.21.15.57:0/3932853541 wait complete. 2023-10-07T23:23:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 Processor -- start 2023-10-07T23:23:11.121 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.117+0000 7ff7f9769700 1 -- start start 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406ac90 0x7ff7f414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f4155db0 0x7ff7f4140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f413ca70 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f41440a0 con 0x7ff7f406ac90 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f414d2e0 con 0x7ff7f4155db0 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38998/0 (socket says 172.21.15.57:38998) 2023-10-07T23:23:11.123 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3659194169 learned_addr learned my addr 172.21.15.57:0/3659194169 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f3fff700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406ac90 0x7ff7f414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f8f68700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f4155db0 0x7ff7f4140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f4155db0 msgr2=0x7ff7f4140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f4155db0 0x7ff7f4140680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406ac90 msgr2=0x7ff7f414f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406ac90 0x7ff7f414f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 -- 172.21.15.57:0/3659194169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80094f0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f37fe700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006fd0 tx=0x7ff7e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800bbe0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f4150cc0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e8000c90 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8011650 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.121+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f41445d0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.126 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f40711c0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff7e8011a00 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.127 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7dbfff700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 0x7ff7e0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:11.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff7e807daa0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.128 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7f3fff700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 0x7ff7e0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:11.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.125+0000 7ff7f3fff700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 0x7ff7e0062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:11.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.133+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff7e804e2f0 con 0x7ff7f40c9ad0 2023-10-07T23:23:11.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:11.353+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) v1 -- 0x7ff7f4161d20 con 0x7ff7f40c9ad0 2023-10-07T23:23:12.056 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.053+0000 7ff7dbfff700 1 -- 172.21.15.57:0/3659194169 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v107) v1 ==== 165+0+0 (secure 0 0 0) 0x7ff7e80150e0 con 0x7ff7f40c9ad0 2023-10-07T23:23:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 msgr2=0x7ff7e0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 0x7ff7e0062a00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4000c00 tx=0x7ff7e4007ea0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 msgr2=0x7ff7f413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006fd0 tx=0x7ff7e8017000 comp rx=0 tx=0).stop 2023-10-07T23:23:12.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 shutdown_connections 2023-10-07T23:23:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff7f4155db0 0x7ff7f4140680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7f40c9ad0 0x7ff7f413c530 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff7e0060550 0x7ff7e0062a00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 --2- 172.21.15.57:0/3659194169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff7f406ac90 0x7ff7f414f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 >> 172.21.15.57:0/3659194169 conn(0x7ff7f4067b80 msgr2=0x7ff7f40b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:12.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 shutdown_connections 2023-10-07T23:23:12.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.057+0000 7ff7f9769700 1 -- 172.21.15.57:0/3659194169 wait complete. 2023-10-07T23:23:12.077 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2023-10-07T23:23:12.077 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T23:23:12.078 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:23:12.078 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-07T23:23:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 Processor -- start 2023-10-07T23:23:12.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 -- start start 2023-10-07T23:23:12.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e00c9a90 0x7f32e015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 0x7f32e006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e0067750 con 0x7f32e00c96c0 2023-10-07T23:23:12.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e00c9f00 con 0x7f32e00c9a90 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e59f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0067d10 con 0x7f32e015cfa0 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32dffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32e00c9a90 0x7f32e015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e51f4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 0x7f32e006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e49f3700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f32e00c96c0 0x7f32e00681a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:58822/0 (socket says 172.21.15.57:58822) 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.193+0000 7f32e49f3700 1 -- 172.21.15.57:0/1514073727 learned_addr learned my addr 172.21.15.57:0/1514073727 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:12.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32df7fe700 1 -- 172.21.15.57:0/1514073727 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353435651 0 0) 0x7f32e0067750 con 0x7f32e00c96c0 2023-10-07T23:23:12.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32df7fe700 1 -- 172.21.15.57:0/1514073727 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d0003570 con 0x7f32e00c96c0 2023-10-07T23:23:12.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32dffff700 1 -- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 msgr2=0x7f32e006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:12.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32dffff700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 0x7f32e006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32dffff700 1 -- 172.21.15.57:0/1514073727 >> v1:172.21.15.154:6789/0 conn(0x7f32e00c96c0 legacy=0x7f32e00681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32dffff700 1 -- 172.21.15.57:0/1514073727 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e015d500 con 0x7f32e00c9a90 2023-10-07T23:23:12.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e51f4700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 0x7f32e006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:12.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32dffff700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e00c9a90 0x7f32e015ca60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0098e0 tx=0x7f32cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c21c6af1a6f3d5e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:12.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32df7fe700 1 -- 172.21.15.57:0/1514073727 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc011630 con 0x7f32e00c9a90 2023-10-07T23:23:12.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32df7fe700 1 -- 172.21.15.57:0/1514073727 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc011790 con 0x7f32e00c9a90 2023-10-07T23:23:12.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32df7fe700 1 -- 172.21.15.57:0/1514073727 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc011b30 con 0x7f32e00c9a90 2023-10-07T23:23:12.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 -- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e00c9a90 msgr2=0x7f32e015ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e00c9a90 0x7f32e015ca60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0098e0 tx=0x7f32cc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 -- 172.21.15.57:0/1514073727 shutdown_connections 2023-10-07T23:23:12.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e015cfa0 0x7f32e006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1514073727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e00c9a90 0x7f32e015ca60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 -- 172.21.15.57:0/1514073727 >> 172.21.15.57:0/1514073727 conn(0x7f32e00692f0 msgr2=0x7f32e00696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:12.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 -- 172.21.15.57:0/1514073727 shutdown_connections 2023-10-07T23:23:12.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.197+0000 7f32e59f5700 1 -- 172.21.15.57:0/1514073727 wait complete. 2023-10-07T23:23:12.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 Processor -- start 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- start start 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e00c96c0 0x7f32e015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e015cfa0 0x7f32e01421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0151a90 con 0x7f32e00c9a90 2023-10-07T23:23:12.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0152e90 con 0x7f32e015cfa0 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e01525c0 con 0x7f32e00c96c0 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39008/0 (socket says 172.21.15.57:39008) 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 -- 172.21.15.57:0/1060858200 learned_addr learned my addr 172.21.15.57:0/1060858200 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e51f4700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e015cfa0 0x7f32e01421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e49f3700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e00c96c0 0x7f32e015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 -- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e00c96c0 msgr2=0x7f32e015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e00c96c0 0x7f32e015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 -- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e015cfa0 msgr2=0x7f32e01421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e015cfa0 0x7f32e01421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 -- 172.21.15.57:0/1060858200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8009a20 con 0x7f32e00c9a90 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dffff700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009ed0 tx=0x7f32cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:12.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc011900 con 0x7f32e00c9a90 2023-10-07T23:23:12.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc011a60 con 0x7f32e00c9a90 2023-10-07T23:23:12.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc009420 con 0x7f32e00c9a90 2023-10-07T23:23:12.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc025620 con 0x7f32e00c9a90 2023-10-07T23:23:12.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.201+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e0152840 con 0x7f32e00c9a90 2023-10-07T23:23:12.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.205+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0153040 con 0x7f32e00c9a90 2023-10-07T23:23:12.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.205+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f32cc025a10 con 0x7f32e00c9a90 2023-10-07T23:23:12.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.205+0000 7f32dcff9700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 0x7f32c4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:12.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.205+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f32cc07dba0 con 0x7f32e00c9a90 2023-10-07T23:23:12.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.205+0000 7f32e49f3700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 0x7f32c4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:12.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.209+0000 7f32e49f3700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 0x7f32c4062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d4007fd0 tx=0x7f32d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:12.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.213+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f32e0153040 con 0x7f32e00c9a90 2023-10-07T23:23:12.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:12.429+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7f32e0153040 con 0x7f32e00c9a90 2023-10-07T23:23:13.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.057+0000 7f32dcff9700 1 -- 172.21.15.57:0/1060858200 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v108) v1 ==== 165+0+0 (secure 0 0 0) 0x7f32e0153040 con 0x7f32e00c9a90 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 msgr2=0x7f32c4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 0x7f32c4062a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32d4007fd0 tx=0x7f32d4009040 comp rx=0 tx=0).stop 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 msgr2=0x7f32e00b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009ed0 tx=0x7f32cc017000 comp rx=0 tx=0).stop 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 shutdown_connections 2023-10-07T23:23:13.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f32c4060550 0x7f32c4062a00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32e015cfa0 0x7f32e01421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32e00c9a90 0x7f32e00b3240 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 --2- 172.21.15.57:0/1060858200 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32e00c96c0 0x7f32e015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 >> 172.21.15.57:0/1060858200 conn(0x7f32e00692f0 msgr2=0x7f32e0069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.061+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 shutdown_connections 2023-10-07T23:23:13.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.065+0000 7f32e59f5700 1 -- 172.21.15.57:0/1060858200 wait complete. 2023-10-07T23:23:13.082 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-07T23:23:13.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 Processor -- start 2023-10-07T23:23:13.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 -- start start 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f52080ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52080c9fc0 con 0x7f52080c8580 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520815d850 con 0x7f52080be7f0 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f520cceb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080cc820 con 0x7f52080c8950 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52077fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f52080c8580 0x7f52080cb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59886/0 (socket says 172.21.15.57:59886) 2023-10-07T23:23:13.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52077fe700 1 -- 172.21.15.57:0/2284664889 learned_addr learned my addr 172.21.15.57:0/2284664889 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:13.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f52080ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786274469 0 0) 0x7f52080c9fc0 con 0x7f52080c8580 2023-10-07T23:23:13.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003570 con 0x7f52080c8580 2023-10-07T23:23:13.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591790211 0 0) 0x7f51f0003570 con 0x7f52080c8580 2023-10-07T23:23:13.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 msgr2=0x7f52080ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f52080ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 >> v1:172.21.15.154:6789/0 conn(0x7f52080be7f0 legacy=0x7f52080bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080cbfd0 con 0x7f52080c8580 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f51fc001470 con 0x7f52080c8580 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 >> v1:172.21.15.57:6789/0 conn(0x7f52080c8580 legacy=0x7f52080cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51f0003a90 0x7f51f0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f0004810 0x7f51f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f51f00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080cc820 con 0x7f51f0003a90 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000bb60 con 0x7f51f0004810 2023-10-07T23:23:13.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000bd20 con 0x7f52080c8950 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f0004810 0x7f51f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f52077fe700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51f0003a90 0x7f51f0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5207fff700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f51f00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 -- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 msgr2=0x7f51f00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080c8950 0x7f51f00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 -- 172.21.15.57:0/2284664889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51f0003a90 msgr2=0x7f51f0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f51f0003a90 0x7f51f0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.197+0000 7f5206ffd700 1 -- 172.21.15.57:0/2284664889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0003570 con 0x7f51f0004810 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f0004810 0x7f51f0004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009c00 tx=0x7f51f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ca1abf6e1c10d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800e710 con 0x7f51f0004810 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f800e870 con 0x7f51f0004810 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f52067fc700 1 -- 172.21.15.57:0/2284664889 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800ec30 con 0x7f51f0004810 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 -- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f0004810 msgr2=0x7f51f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 --2- 172.21.15.57:0/2284664889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51f0004810 0x7f51f0004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009c00 tx=0x7f51f80091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f5207fff700 1 -- 172.21.15.57:0/2284664889 reap_dead start 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 -- 172.21.15.57:0/2284664889 shutdown_connections 2023-10-07T23:23:13.205 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 -- 172.21.15.57:0/2284664889 >> 172.21.15.57:0/2284664889 conn(0x7f52080670e0 msgr2=0x7f52080674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 -- 172.21.15.57:0/2284664889 shutdown_connections 2023-10-07T23:23:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.201+0000 7f520cceb700 1 -- 172.21.15.57:0/2284664889 wait complete. 2023-10-07T23:23:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 Processor -- start 2023-10-07T23:23:13.206 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 -- start start 2023-10-07T23:23:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080aab20 0x7f520816e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5208171810 0x7f52080b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080abe00 con 0x7f52080c6280 2023-10-07T23:23:13.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080b1380 con 0x7f5208171810 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080b0e70 con 0x7f52080aab20 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39022/0 (socket says 172.21.15.57:39022) 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 -- 172.21.15.57:0/2681919921 learned_addr learned my addr 172.21.15.57:0/2681919921 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5206ffd700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5208171810 0x7f52080b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f52077fe700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080aab20 0x7f520816e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 -- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080aab20 msgr2=0x7f520816e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080aab20 0x7f520816e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 -- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5208171810 msgr2=0x7f52080b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5208171810 0x7f52080b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 -- 172.21.15.57:0/2681919921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080c6c30 con 0x7f52080c6280 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f5207fff700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f52080b1b30 tx=0x7f5200009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000032e0 con 0x7f52080c6280 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200001580 con 0x7f52080c6280 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52080b10a0 con 0x7f52080c6280 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.205+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200003440 con 0x7f52080c6280 2023-10-07T23:23:13.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5208171010 con 0x7f52080c6280 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5208170b90 con 0x7f52080c6280 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5200001b30 con 0x7f52080c6280 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f51ef7fe700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 0x7f51f4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f520007bd50 con 0x7f52080c6280 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.209+0000 7f52077fe700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 0x7f51f4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.213+0000 7f52077fe700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 0x7f51f4062a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f51fc007fb0 tx=0x7f51fc001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.221+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5208170b90 con 0x7f52080c6280 2023-10-07T23:23:13.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.381+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5208170b90 con 0x7f52080c6280 2023-10-07T23:23:13.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f51ef7fe700 1 -- 172.21.15.57:0/2681919921 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5208170b90 con 0x7f52080c6280 2023-10-07T23:23:13.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 msgr2=0x7f51f4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 0x7f51f4062a00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f51fc007fb0 tx=0x7f51fc001e50 comp rx=0 tx=0).stop 2023-10-07T23:23:13.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 msgr2=0x7f52080c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f52080b1b30 tx=0x7f5200009040 comp rx=0 tx=0).stop 2023-10-07T23:23:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 shutdown_connections 2023-10-07T23:23:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52080c6280 0x7f52080c66f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f51f4060550 0x7f51f4062a00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5208171810 0x7f52080b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 --2- 172.21.15.57:0/2681919921 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52080aab20 0x7f520816e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 >> 172.21.15.57:0/2681919921 conn(0x7f52080670e0 msgr2=0x7f5208070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 shutdown_connections 2023-10-07T23:23:13.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.393+0000 7f520cceb700 1 -- 172.21.15.57:0/2681919921 wait complete. 2023-10-07T23:23:13.411 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T23:23:13.412 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:23:13.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:23:13.449 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T23:23:13.450 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-07T23:23:13.451 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 Processor -- start 2023-10-07T23:23:13.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 -- start start 2023-10-07T23:23:13.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e7006bb00 0x7f6e7015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7015ccb0 con 0x7f6e700aefb0 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7015cf70 con 0x7f6e7006cf10 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e762ce700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7015d250 con 0x7f6e7006bb00 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6ffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6e700aefb0 0x7f6e7006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59900/0 (socket says 172.21.15.57:59900) 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6ffff700 1 -- 172.21.15.57:0/3397598515 learned_addr learned my addr 172.21.15.57:0/3397598515 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e7486b700 1 --2- 172.21.15.57:0/3397598515 >> v2:172.21.15.154:3300/0 conn(0x7f6e7006bb00 0x7f6e7015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485058760 0 0) 0x7f6e7015ccb0 con 0x7f6e700aefb0 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e50003570 con 0x7f6e700aefb0 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4197305903 0 0) 0x7f6e50003570 con 0x7f6e700aefb0 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 >> v2:172.21.15.154:3300/0 conn(0x7f6e7006bb00 msgr2=0x7f6e7015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 --2- 172.21.15.57:0/3397598515 >> v2:172.21.15.154:3300/0 conn(0x7f6e7006bb00 0x7f6e7015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 >> v1:172.21.15.154:6789/0 conn(0x7f6e7006cf10 legacy=0x7f6e701599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7015d530 con 0x7f6e700aefb0 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f6e680017f0 con 0x7f6e700aefb0 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 >> v1:172.21.15.57:6789/0 conn(0x7f6e700aefb0 legacy=0x7f6e7006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50003a60 0x7f6e50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e500052e0 0x7f6e5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7015d250 con 0x7f6e50004810 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5000c190 con 0x7f6e50003a60 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5000c350 con 0x7f6e500052e0 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e7486b700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e500052e0 0x7f6e5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6ffff700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50003a60 0x7f6e50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 -- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e500052e0 msgr2=0x7f6e5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e500052e0 0x7f6e5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 -- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50003a60 msgr2=0x7f6e50003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e50003a60 0x7f6e50003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e7486b700 1 -- 172.21.15.57:0/3397598515 reap_dead start 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.581+0000 7f6e6f7fe700 1 -- 172.21.15.57:0/3397598515 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50003570 con 0x7f6e50004810 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e50004c80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e580018b0 tx=0x7f6e58007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79960a74ecf148f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58012420 con 0x7f6e50004810 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e580125a0 con 0x7f6e50004810 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e6effd700 1 -- 172.21.15.57:0/3397598515 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58012960 con 0x7f6e50004810 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 -- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 msgr2=0x7f6e50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e50004c80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e580018b0 tx=0x7f6e58007d30 comp rx=0 tx=0).stop 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 -- 172.21.15.57:0/3397598515 shutdown_connections 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 --2- 172.21.15.57:0/3397598515 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e50004c80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 -- 172.21.15.57:0/3397598515 >> 172.21.15.57:0/3397598515 conn(0x7f6e70068630 msgr2=0x7f6e70068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 -- 172.21.15.57:0/3397598515 shutdown_connections 2023-10-07T23:23:13.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.585+0000 7f6e762ce700 1 -- 172.21.15.57:0/3397598515 wait complete. 2023-10-07T23:23:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 Processor -- start 2023-10-07T23:23:13.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 -- start start 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7016c610 0x7f6e700c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e700c7090 0x7f6e700c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70150220 con 0x7f6e50004810 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70155030 con 0x7f6e700c7090 2023-10-07T23:23:13.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e762ce700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7015c9e0 con 0x7f6e7016c610 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39042/0 (socket says 172.21.15.57:39042) 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 -- 172.21.15.57:0/1226425459 learned_addr learned my addr 172.21.15.57:0/1226425459 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7016c610 0x7f6e700c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e7486b700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e700c7090 0x7f6e700c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 -- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7016c610 msgr2=0x7f6e700c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7016c610 0x7f6e700c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 -- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e700c7090 msgr2=0x7f6e700c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e700c7090 0x7f6e700c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.589+0000 7f6e6ffff700 1 -- 172.21.15.57:0/1226425459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e700c7590 con 0x7f6e50004810 2023-10-07T23:23:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e6ffff700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007ac0 tx=0x7f6e680021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800acc0 con 0x7f6e50004810 2023-10-07T23:23:13.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800ae40 con 0x7f6e50004810 2023-10-07T23:23:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e700c7870 con 0x7f6e50004810 2023-10-07T23:23:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68013640 con 0x7f6e50004810 2023-10-07T23:23:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700c7e30 con 0x7f6e50004810 2023-10-07T23:23:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e701551c0 con 0x7f6e50004810 2023-10-07T23:23:13.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6e680139f0 con 0x7f6e50004810 2023-10-07T23:23:13.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 0x7f6e54062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6e6807cfe0 con 0x7f6e50004810 2023-10-07T23:23:13.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.593+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 0x7f6e54062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.597+0000 7f6e6f7fe700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 0x7f6e54062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e7015a700 tx=0x7f6e58015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.605+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6e6804d8d0 con 0x7f6e50004810 2023-10-07T23:23:13.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e700711c0 con 0x7f6e50004810 2023-10-07T23:23:13.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e67fff700 1 -- 172.21.15.57:0/1226425459 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f6e68050ef0 con 0x7f6e50004810 2023-10-07T23:23:13.803 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:13.803 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/2960397297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2960397297},{"type":"v1","addr":"172.21.15.154:6836","nonce":2960397297}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4748,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/433555014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":433555014},{"type":"v1","addr":"172.21.15.154:6837","nonce":433555014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4953,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/1785162546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":1785162546},{"type":"v1","addr":"172.21.15.57:6833","nonce":1785162546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/311273341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":311273341},{"type":"v1","addr":"172.21.15.57:6835","nonce":311273341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[]} 2023-10-07T23:23:13.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 msgr2=0x7f6e54062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 0x7f6e54062a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6e7015a700 tx=0x7f6e58015000 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 msgr2=0x7f6e7015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.801+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007ac0 tx=0x7f6e680021d0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 shutdown_connections 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6e54060550 0x7f6e54062a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e700c7090 0x7f6e700c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6e7016c610 0x7f6e700c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 --2- 172.21.15.57:0/1226425459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e50004810 0x7f6e7015c4a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 >> 172.21.15.57:0/1226425459 conn(0x7f6e70068630 msgr2=0x7f6e70150cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 shutdown_connections 2023-10-07T23:23:13.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.805+0000 7f6e762ce700 1 -- 172.21.15.57:0/1226425459 wait complete. 2023-10-07T23:23:13.807 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 15 2023-10-07T23:23:13.818 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-07T23:23:13.818 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-07T23:23:13.818 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T23:23:13.818 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:23:13.818 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-07T23:23:13.819 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-07T23:23:13.819 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T23:23:13.819 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:23:13.819 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-07T23:23:13.819 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-07T23:23:13.819 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T23:23:13.819 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:23:13.820 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-07T23:23:13.820 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-07T23:23:13.820 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T23:23:13.820 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:23:13.829 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T23:23:13.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-07T23:23:13.832 INFO:tasks.ceph.mds.c:Started 2023-10-07T23:23:13.832 DEBUG:teuthology.parallel:result is None 2023-10-07T23:23:13.833 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T23:23:13.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-07T23:23:13.835 INFO:tasks.ceph.mds.a:Started 2023-10-07T23:23:13.835 DEBUG:teuthology.parallel:result is None 2023-10-07T23:23:13.836 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T23:23:13.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-07T23:23:13.837 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T23:23:13.837 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-07T23:23:13.881 INFO:tasks.ceph.mds.d:Started 2023-10-07T23:23:13.881 DEBUG:teuthology.parallel:result is None 2023-10-07T23:23:13.882 INFO:tasks.ceph.mds.b:Started 2023-10-07T23:23:13.882 DEBUG:teuthology.parallel:result is None 2023-10-07T23:23:13.882 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-07T23:23:13.903 INFO:tasks.ceph.mds.a.smithi057.stdout:starting mds.a at 2023-10-07T23:23:13.909 INFO:tasks.ceph.mds.c.smithi057.stdout:starting mds.c at 2023-10-07T23:23:13.957 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-07T23:23:13.960 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2023-10-07T23:23:13.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.981+0000 7f8af8caa700 1 Processor -- start 2023-10-07T23:23:13.985 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.981+0000 7f8af8caa700 1 -- start start 2023-10-07T23:23:13.988 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 0x7f8af40cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8af415a0a0 0x7f8af415a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af4159930 con 0x7f8af406c800 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af4159c10 con 0x7f8af4155a60 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af4156bd0 con 0x7f8af415a0a0 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af259c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 0x7f8af40cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af2d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8af406c800 0x7f8af40c8180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59924/0 (socket says 172.21.15.57:59924) 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af259c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 0x7f8af40cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39090/0 (socket says 172.21.15.57:39090) 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af2d9d700 1 -- 172.21.15.57:0/412526627 learned_addr learned my addr 172.21.15.57:0/412526627 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:13.989 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/412526627 >> v2:172.21.15.154:3300/0 conn(0x7f8af415a0a0 0x7f8af415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 -- 172.21.15.57:0/412526627 >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 msgr2=0x7f8af40cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/412526627 >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 0x7f8af40cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 -- 172.21.15.57:0/412526627 >> v1:172.21.15.57:6789/0 conn(0x7f8af406c800 legacy=0x7f8af40c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 -- 172.21.15.57:0/412526627 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4156eb0 con 0x7f8af415a0a0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/412526627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af415a510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8af406c550 tx=0x7f8ae80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9a71fdf8fb303 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af159a700 1 -- 172.21.15.57:0/412526627 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8011630 con 0x7f8af415a0a0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af159a700 1 -- 172.21.15.57:0/412526627 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8011790 con 0x7f8af415a0a0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af159a700 1 -- 172.21.15.57:0/412526627 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8011b30 con 0x7f8af415a0a0 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- 172.21.15.57:0/412526627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 msgr2=0x7f8af415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 --2- 172.21.15.57:0/412526627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af415a510 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8af406c550 tx=0x7f8ae80091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- 172.21.15.57:0/412526627 shutdown_connections 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 --2- 172.21.15.57:0/412526627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af415a510 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 --2- 172.21.15.57:0/412526627 >> v2:172.21.15.57:3300/0 conn(0x7f8af4155a60 0x7f8af40cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- 172.21.15.57:0/412526627 >> 172.21.15.57:0/412526627 conn(0x7f8af40ccef0 msgr2=0x7f8af40c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- 172.21.15.57:0/412526627 shutdown_connections 2023-10-07T23:23:13.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.985+0000 7f8af8caa700 1 -- 172.21.15.57:0/412526627 wait complete. 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 Processor -- start 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- start start 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af406c800 0x7f8af4159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af406cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41595c0 con 0x7f8af4155a60 2023-10-07T23:23:13.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40ca1d0 con 0x7f8af415a0a0 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af40ca4b0 con 0x7f8af406c800 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39096/0 (socket says 172.21.15.57:39096) 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 -- 172.21.15.57:0/756584425 learned_addr learned my addr 172.21.15.57:0/756584425 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af2d9d700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af406cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af259c700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af406c800 0x7f8af4159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 -- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af406c800 msgr2=0x7f8af4159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af406c800 0x7f8af4159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 -- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 msgr2=0x7f8af406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af406cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 -- 172.21.15.57:0/756584425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8009420 con 0x7f8af4155a60 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af1d9b700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8af4156d10 tx=0x7f8ae8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8000d10 con 0x7f8af4155a60 2023-10-07T23:23:13.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8011b10 con 0x7f8af4155a60 2023-10-07T23:23:13.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af40ca790 con 0x7f8af4155a60 2023-10-07T23:23:13.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8025660 con 0x7f8af4155a60 2023-10-07T23:23:13.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.989+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af40cad50 con 0x7f8af4155a60 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.993+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af40caf40 con 0x7f8af4155a60 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.993+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8ae8025a90 con 0x7f8af4155a60 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.993+0000 7f8ae27fc700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 0x7f8ad4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.993+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8ae807dd80 con 0x7f8af4155a60 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.993+0000 7f8af259c700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 0x7f8ad4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:13.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:13.997+0000 7f8af259c700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 0x7f8ad4062a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009d20 tx=0x7f8ae40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:14.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:14.001+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8ae804e5d0 con 0x7f8af4155a60 2023-10-07T23:23:14.187 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:14.185+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8af40647b0 con 0x7f8af4155a60 2023-10-07T23:23:15.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.093+0000 7f8ae27fc700 1 -- 172.21.15.57:0/756584425 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ae8015020 con 0x7f8af4155a60 2023-10-07T23:23:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.097+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 msgr2=0x7f8ad4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.097+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 0x7f8ad4062a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009d20 tx=0x7f8ae40091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.097+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 msgr2=0x7f8af414dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.097+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8af4156d10 tx=0x7f8ae8017000 comp rx=0 tx=0).stop 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 shutdown_connections 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8ad4060550 0x7f8ad4062a00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8af415a0a0 0x7f8af406cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8af4155a60 0x7f8af414dfa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 --2- 172.21.15.57:0/756584425 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8af406c800 0x7f8af4159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 >> 172.21.15.57:0/756584425 conn(0x7f8af40ccef0 msgr2=0x7f8af40c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 shutdown_connections 2023-10-07T23:23:15.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.101+0000 7f8af8caa700 1 -- 172.21.15.57:0/756584425 wait complete. 2023-10-07T23:23:15.103 INFO:teuthology.orchestra.run.smithi057.stderr: removed all blocklist entries 2023-10-07T23:23:15.118 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-07T23:23:15.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.277+0000 7efe9968b700 1 Processor -- start 2023-10-07T23:23:15.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.277+0000 7efe9968b700 1 -- start start 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe9968b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe9968b700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9406d990 con 0x7efe9406c800 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe9968b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9406cd80 con 0x7efe94158ee0 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe9968b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9406d060 con 0x7efe941592b0 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe98e8a700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7efe9406c800 0x7efe94157380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59930/0 (socket says 172.21.15.57:59930) 2023-10-07T23:23:15.283 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe98e8a700 1 -- 172.21.15.57:0/961481327 learned_addr learned my addr 172.21.15.57:0/961481327 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:15.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe8bfff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950459698 0 0) 0x7efe9406d990 con 0x7efe9406c800 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe80003570 con 0x7efe9406c800 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809451027 0 0) 0x7efe9406cd80 con 0x7efe94158ee0 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9406d990 con 0x7efe94158ee0 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4202718274 0 0) 0x7efe80003570 con 0x7efe9406c800 2023-10-07T23:23:15.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 msgr2=0x7efe940c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe940c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 >> v1:172.21.15.154:6789/0 conn(0x7efe94158ee0 legacy=0x7efe94156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9406d340 con 0x7efe9406c800 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7efe8c0017f0 con 0x7efe9406c800 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe8bfff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe940c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 >> v1:172.21.15.57:6789/0 conn(0x7efe9406c800 legacy=0x7efe94157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe80004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe80004810 0x7efe80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe80005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9406d060 con 0x7efe80003c20 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8000bbf0 con 0x7efe80004810 2023-10-07T23:23:15.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe93fff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe80004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8000dfc0 con 0x7efe941592b0 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe8bfff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe80004810 0x7efe80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.281+0000 7efe98e8a700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe80005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 -- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 msgr2=0x7efe80005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941592b0 0x7efe80005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 -- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe80004810 msgr2=0x7efe80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe80004810 0x7efe80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 -- 172.21.15.57:0/961481327 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c009420 con 0x7efe80003c20 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe8bfff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe80004810 0x7efe80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe93fff700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe80004010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe840019e0 tx=0x7efe84007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aef382eb2e6a7de server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400c5d0 con 0x7efe80003c20 2023-10-07T23:23:15.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8400c730 con 0x7efe80003c20 2023-10-07T23:23:15.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe937fe700 1 -- 172.21.15.57:0/961481327 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400cad0 con 0x7efe80003c20 2023-10-07T23:23:15.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- 172.21.15.57:0/961481327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 msgr2=0x7efe80004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 --2- 172.21.15.57:0/961481327 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe80004010 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe840019e0 tx=0x7efe84007d30 comp rx=0 tx=0).stop 2023-10-07T23:23:15.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe98e8a700 1 -- 172.21.15.57:0/961481327 reap_dead start 2023-10-07T23:23:15.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- 172.21.15.57:0/961481327 shutdown_connections 2023-10-07T23:23:15.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- 172.21.15.57:0/961481327 >> 172.21.15.57:0/961481327 conn(0x7efe94067b80 msgr2=0x7efe940c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- 172.21.15.57:0/961481327 shutdown_connections 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- 172.21.15.57:0/961481327 wait complete. 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 Processor -- start 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.285+0000 7efe9968b700 1 -- start start 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe94157ad0 0x7efe94151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941517b0 0x7efe9414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940cb0c0 con 0x7efe80003c20 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940ca4b0 con 0x7efe94157ad0 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940ca790 con 0x7efe941517b0 2023-10-07T23:23:15.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39106/0 (socket says 172.21.15.57:39106) 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 -- 172.21.15.57:0/679227346 learned_addr learned my addr 172.21.15.57:0/679227346 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe98e8a700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941517b0 0x7efe9414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe8bfff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe94157ad0 0x7efe94151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 -- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941517b0 msgr2=0x7efe9414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941517b0 0x7efe9414a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 -- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe94157ad0 msgr2=0x7efe94151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe94157ad0 0x7efe94151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 -- 172.21.15.57:0/679227346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940caa70 con 0x7efe80003c20 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe93fff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe84012d10 tx=0x7efe84015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84023070 con 0x7efe80003c20 2023-10-07T23:23:15.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84000fe0 con 0x7efe80003c20 2023-10-07T23:23:15.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940cad50 con 0x7efe80003c20 2023-10-07T23:23:15.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400c5d0 con 0x7efe80003c20 2023-10-07T23:23:15.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9414aea0 con 0x7efe80003c20 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940711c0 con 0x7efe80003c20 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.289+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efe84019070 con 0x7efe80003c20 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.293+0000 7efe90ff9700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 0x7efe74062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.293+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efe8407d1f0 con 0x7efe80003c20 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.293+0000 7efe8bfff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 0x7efe74062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.293+0000 7efe8bfff700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 0x7efe74062a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe940c7eb0 tx=0x7efe7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.301 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.301+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efe8404dbf0 con 0x7efe80003c20 2023-10-07T23:23:15.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efe940cb850 con 0x7efe80003c20 2023-10-07T23:23:15.522 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe90ff9700 1 -- 172.21.15.57:0/679227346 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efe84017070 con 0x7efe80003c20 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: "auth_dump": [ 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.a", 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFl4EJHMBAAHkCAWP6+9FdWp9FusnpX3w==", 2023-10-07T23:23:15.523 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.524 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.b", 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFldG8xNxAAj8EFNonJE0X5pMyB0izMxQ==", 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.525 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.c", 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlooFlMxAAG+NVHxNzdNg5cRjAtnAOLQ==", 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.526 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mds.d", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFloYdROhAA1y7ZdlNsVUsuaarTEcolVQ==", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.0", 2023-10-07T23:23:15.527 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC65yFlqv5KDhAA25InypbQtn4sjojMQ1q4UA==", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.1", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC95yFl3NxlFRAAZBqGACWZ4uyN0fXNtNtuYQ==", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.528 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.2", 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDA5yFlYBmuGBAAgcTTcujrs1CIHWZWM1o2VQ==", 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.529 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.3", 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDD5yFlwgfQHhAAZMmLKVbUCsBebAOEy1pBcA==", 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.4", 2023-10-07T23:23:15.530 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDJ5yFld6uuDhAARh1yNxrm4Ub+VTrik7tLXg==", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.5", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDM5yFlaPZvHxAAwYp3I5ZpOy5S1nBrajKpew==", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.531 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.6", 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDP5yFlGvyDJRAAOFvXIQkB+6Q0YyaQ9McSYA==", 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.532 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "osd.7", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQDS5yFl445yKxAAh/wYkbCQUJy+nmP4S1lsZA==", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow profile osd", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile osd", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.0", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlzElqBBAAvQsOKVpmlkOCrZOBdS/Mng==", 2023-10-07T23:23:15.533 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow r", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.1", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlydslChAAcyKLUVIvuQ8YjyJ+J0Lepw==", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:23:15.534 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.2", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlZJ0YBxAAbk4yS+Sl1PXtOnTxzFikDg==", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.3", 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQC05yFlS/3JDxAADLDbTFyercFRlld7VfwQEA==", 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow", 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow r", 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow rw", 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow rwx" 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.536 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "client.admin", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlzLRqIBAASar5Wgxeyp1nifhwLMYkmw==", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr": "allow *", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow *", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.x", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlB/VhLRAAld/Nnm46+mosax2vvzWOJQ==", 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.537 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "entity": "mgr.y", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "key": "AQCz5yFlDPSDKhAALtUKYEj0TqzdoaXF8L+f/Q==", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": { 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": "allow *", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "mon": "allow profile mgr", 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: "osd": "allow *" 2023-10-07T23:23:15.538 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.539 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:15.539 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2023-10-07T23:23:15.539 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:23:15.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 msgr2=0x7efe74062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 0x7efe74062a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efe940c7eb0 tx=0x7efe7c008040 comp rx=0 tx=0).stop 2023-10-07T23:23:15.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 msgr2=0x7efe94157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe84012d10 tx=0x7efe84015000 comp rx=0 tx=0).stop 2023-10-07T23:23:15.540 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 shutdown_connections 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe941517b0 0x7efe9414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe74060550 0x7efe74062a00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe94157ad0 0x7efe94151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 --2- 172.21.15.57:0/679227346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe80003c20 0x7efe94157590 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 >> 172.21.15.57:0/679227346 conn(0x7efe94067b80 msgr2=0x7efe940c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 shutdown_connections 2023-10-07T23:23:15.541 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.521+0000 7efe9968b700 1 -- 172.21.15.57:0/679227346 wait complete. 2023-10-07T23:23:15.542 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:15.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 Processor -- start 2023-10-07T23:23:15.650 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- start start 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 0x7f9e840c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9e840c8570 0x7f9e840caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e84068590 con 0x7f9e84069220 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e84069660 con 0x7f9e840c8570 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e840c74d0 con 0x7f9e840cab70 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9e840c8570 0x7f9e840caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e89950700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f9e84069220 0x7f9e8415ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:59944/0 (socket says 172.21.15.57:59944) 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f9e840c8570 0x7f9e840caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39112/0 (socket says 172.21.15.57:39112) 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e89950700 1 -- 172.21.15.57:0/1192706233 learned_addr learned my addr 172.21.15.57:0/1192706233 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:15.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8914f700 1 --2- 172.21.15.57:0/1192706233 >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 0x7f9e840c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 -- 172.21.15.57:0/1192706233 >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 msgr2=0x7f9e840c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 --2- 172.21.15.57:0/1192706233 >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 0x7f9e840c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 -- 172.21.15.57:0/1192706233 >> v1:172.21.15.57:6789/0 conn(0x7f9e84069220 legacy=0x7f9e8415ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 -- 172.21.15.57:0/1192706233 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e840c77b0 con 0x7f9e840c8570 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8914f700 1 --2- 172.21.15.57:0/1192706233 >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 0x7f9e840c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8894e700 1 --2- 172.21.15.57:0/1192706233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840c8570 0x7f9e840caf40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009e60 tx=0x7f9e7c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465c620d9ce2d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e77fff700 1 -- 172.21.15.57:0/1192706233 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00e070 con 0x7f9e840c8570 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e77fff700 1 -- 172.21.15.57:0/1192706233 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c00bbf0 con 0x7f9e840c8570 2023-10-07T23:23:15.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e77fff700 1 -- 172.21.15.57:0/1192706233 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c012430 con 0x7f9e840c8570 2023-10-07T23:23:15.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/1192706233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840c8570 msgr2=0x7f9e840caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/1192706233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840c8570 0x7f9e840caf40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009e60 tx=0x7f9e7c009780 comp rx=0 tx=0).stop 2023-10-07T23:23:15.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/1192706233 shutdown_connections 2023-10-07T23:23:15.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/1192706233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840c8570 0x7f9e840caf40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/1192706233 >> v2:172.21.15.154:3300/0 conn(0x7f9e840cab70 0x7f9e840c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/1192706233 >> 172.21.15.57:0/1192706233 conn(0x7f9e840cce30 msgr2=0x7f9e840c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/1192706233 shutdown_connections 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.649+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/1192706233 wait complete. 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 Processor -- start 2023-10-07T23:23:15.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- start start 2023-10-07T23:23:15.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 0x7f9e840c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e840c8570 0x7f9e840c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840c62c0 con 0x7f9e840cab70 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e84157f10 con 0x7f9e840c8570 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840c44b0 con 0x7f9e84069220 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39120/0 (socket says 172.21.15.57:39120) 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 -- 172.21.15.57:0/2535645872 learned_addr learned my addr 172.21.15.57:0/2535645872 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8894e700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e840c8570 0x7f9e840c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8914f700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 0x7f9e840c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 -- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 msgr2=0x7f9e840c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 0x7f9e840c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 -- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e840c8570 msgr2=0x7f9e840c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e840c8570 0x7f9e840c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 -- 172.21.15.57:0/2535645872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c009a20 con 0x7f9e840cab70 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8914f700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 0x7f9e840c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e89950700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c590 tx=0x7f9e80007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80005b30 con 0x7f9e840cab70 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80005cb0 con 0x7f9e840cab70 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8415adb0 con 0x7f9e840cab70 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80007320 con 0x7f9e840cab70 2023-10-07T23:23:15.659 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.653+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e840c3b80 con 0x7f9e840cab70 2023-10-07T23:23:15.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.657+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e840711c0 con 0x7f9e840cab70 2023-10-07T23:23:15.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.657+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9e8001b3b0 con 0x7f9e840cab70 2023-10-07T23:23:15.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.657+0000 7f9e757fa700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 0x7f9e68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.657+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9e8007ba20 con 0x7f9e840cab70 2023-10-07T23:23:15.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.657+0000 7f9e8914f700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 0x7f9e68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.661 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.661+0000 7f9e8914f700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 0x7f9e68062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.667 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.665+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9e8004c2c0 con 0x7f9e840cab70 2023-10-07T23:23:15.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.829+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e840c45f0 con 0x7f9e840cab70 2023-10-07T23:23:15.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.829+0000 7f9e757fa700 1 -- 172.21.15.57:0/2535645872 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f9e80017090 con 0x7f9e840cab70 2023-10-07T23:23:15.832 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:15.832 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":43,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:23:15.088761+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:15.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 msgr2=0x7f9e68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 0x7f9e68062a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78008040 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 msgr2=0x7f9e841579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c590 tx=0x7f9e80007ef0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 shutdown_connections 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9e840cab70 0x7f9e841579d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9e68060550 0x7f9e68062a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e840c8570 0x7f9e840c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 --2- 172.21.15.57:0/2535645872 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9e84069220 0x7f9e840c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 >> 172.21.15.57:0/2535645872 conn(0x7f9e840cce30 msgr2=0x7f9e841626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 shutdown_connections 2023-10-07T23:23:15.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.833+0000 7f9e8b3b3700 1 -- 172.21.15.57:0/2535645872 wait complete. 2023-10-07T23:23:15.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-07T23:23:15.846 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:23:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.961+0000 7ff68bfff700 1 Processor -- start 2023-10-07T23:23:15.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.961+0000 7ff68bfff700 1 -- start start 2023-10-07T23:23:15.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c155790 0x7ff68c069c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff68c15a130 0x7ff68c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c159ca0 con 0x7ff68c0c8600 2023-10-07T23:23:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c15a580 con 0x7ff68c15a130 2023-10-07T23:23:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c159670 con 0x7ff68c155790 2023-10-07T23:23:15.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff68c15a130 0x7ff68c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c155790 0x7ff68c069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff68c15a130 0x7ff68c155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:37090/0 (socket says 172.21.15.57:37090) 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 -- 172.21.15.57:0/3250444552 learned_addr learned my addr 172.21.15.57:0/3250444552 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 -- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c155790 msgr2=0x7ff68c069c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 --2- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c155790 0x7ff68c069c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 -- 172.21.15.57:0/3250444552 >> v1:172.21.15.154:6789/0 conn(0x7ff68c0c8600 legacy=0x7ff68c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 -- 172.21.15.57:0/3250444552 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0c8ee0 con 0x7ff68c15a130 2023-10-07T23:23:15.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff6827fc700 1 --2- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c155b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6740099b0 tx=0x7ff674009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a53cd563e4d5da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68a7fc700 1 -- 172.21.15.57:0/3250444552 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674009ea0 con 0x7ff68c15a130 2023-10-07T23:23:15.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68a7fc700 1 -- 172.21.15.57:0/3250444552 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400b860 con 0x7ff68c15a130 2023-10-07T23:23:15.971 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68a7fc700 1 -- 172.21.15.57:0/3250444552 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400bc00 con 0x7ff68c15a130 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 msgr2=0x7ff68c155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 --2- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c155b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6740099b0 tx=0x7ff674009250 comp rx=0 tx=0).stop 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- 172.21.15.57:0/3250444552 shutdown_connections 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 --2- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c155b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 --2- 172.21.15.57:0/3250444552 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c155790 0x7ff68c069c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- 172.21.15.57:0/3250444552 >> 172.21.15.57:0/3250444552 conn(0x7ff68c06b5d0 msgr2=0x7ff68c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- 172.21.15.57:0/3250444552 shutdown_connections 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.965+0000 7ff68bfff700 1 -- 172.21.15.57:0/3250444552 wait complete. 2023-10-07T23:23:15.972 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 Processor -- start 2023-10-07T23:23:15.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 -- start start 2023-10-07T23:23:15.975 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 0x7ff68c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c1593d0 con 0x7ff68c155790 2023-10-07T23:23:15.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0b0240 con 0x7ff68c15a130 2023-10-07T23:23:15.976 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68bfff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0c5de0 con 0x7ff68c0c8600 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff6827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff6827fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39132/0 (socket says 172.21.15.57:39132) 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff6827fc700 1 -- 172.21.15.57:0/1458982990 learned_addr learned my addr 172.21.15.57:0/1458982990 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68affd700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 0x7ff68c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.969+0000 7ff68b7fe700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 -- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 msgr2=0x7ff68c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 0x7ff68c158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 -- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 msgr2=0x7ff68c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 -- 172.21.15.57:0/1458982990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740094f0 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff68affd700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 0x7ff68c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6827fc700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff68c06b240 tx=0x7ff674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674000d10 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400b9d0 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c0c60c0 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674025420 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c0b0500 con 0x7ff68c155790 2023-10-07T23:23:15.978 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c0711c0 con 0x7ff68c155790 2023-10-07T23:23:15.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.973+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff674023070 con 0x7ff68c155790 2023-10-07T23:23:15.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.977+0000 7ff6837fe700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 0x7ff670062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:15.979 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.977+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff67407d8a0 con 0x7ff68c155790 2023-10-07T23:23:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.977+0000 7ff68affd700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 0x7ff670062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:15.980 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.977+0000 7ff68affd700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 0x7ff670062a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff67c011480 tx=0x7ff67c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:15.981+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff67404e1e0 con 0x7ff68c155790 2023-10-07T23:23:16.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.141+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff68c0c91c0 con 0x7ff68c155790 2023-10-07T23:23:16.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.141+0000 7ff6837fe700 1 -- 172.21.15.57:0/1458982990 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff674051800 con 0x7ff68c155790 2023-10-07T23:23:16.144 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:23:16.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 msgr2=0x7ff670062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 0x7ff670062a30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff67c011480 tx=0x7ff67c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 msgr2=0x7ff68c0c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff68c06b240 tx=0x7ff674017000 comp rx=0 tx=0).stop 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 shutdown_connections 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff670060580 0x7ff670062a30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff68c15a130 0x7ff68c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff68c155790 0x7ff68c0c54e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 --2- 172.21.15.57:0/1458982990 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff68c0c8600 0x7ff68c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 >> 172.21.15.57:0/1458982990 conn(0x7ff68c06b5d0 msgr2=0x7ff68c149650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 shutdown_connections 2023-10-07T23:23:16.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.145+0000 7ff68bfff700 1 -- 172.21.15.57:0/1458982990 wait complete. 2023-10-07T23:23:16.164 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-07T23:23:16.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 Processor -- start 2023-10-07T23:23:16.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 -- start start 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f536c158be0 0x7f536c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f536c158fb0 0x7f536c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c06b850 0x7f536c0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c06b3c0 con 0x7f536c158fb0 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c06a7b0 con 0x7f536c158be0 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f537083b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c06aa90 con 0x7f536c06b850 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f5363fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f536c158fb0 0x7f536c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f5363fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f536c158fb0 0x7f536c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39142/0 (socket says 172.21.15.57:39142) 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f5363fff700 1 -- 172.21.15.57:0/914042518 learned_addr learned my addr 172.21.15.57:0/914042518 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:16.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f536bfff700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c06b850 0x7f536c0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.273+0000 7f536b7fe700 1 --2- 172.21.15.57:0/914042518 >> v2:172.21.15.154:3300/0 conn(0x7f536c158be0 0x7f536c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 -- 172.21.15.57:0/914042518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c06b850 msgr2=0x7f536c0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:16.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c06b850 0x7f536c0cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 -- 172.21.15.57:0/914042518 >> v2:172.21.15.154:3300/0 conn(0x7f536c158be0 msgr2=0x7f536c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 --2- 172.21.15.57:0/914042518 >> v2:172.21.15.154:3300/0 conn(0x7f536c158be0 0x7f536c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 -- 172.21.15.57:0/914042518 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c06ad70 con 0x7f536c158fb0 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f5363fff700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c158fb0 0x7f536c0ce6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f536c069e50 tx=0x7f5354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d75f0d3fe54cb36 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f536affd700 1 -- 172.21.15.57:0/914042518 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400b640 con 0x7f536c158fb0 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f536affd700 1 -- 172.21.15.57:0/914042518 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400b7a0 con 0x7f536c158fb0 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f536affd700 1 -- 172.21.15.57:0/914042518 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400bb60 con 0x7f536c158fb0 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 -- 172.21.15.57:0/914042518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c158fb0 msgr2=0x7f536c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c158fb0 0x7f536c0ce6b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f536c069e50 tx=0x7f5354009250 comp rx=0 tx=0).stop 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 -- 172.21.15.57:0/914042518 shutdown_connections 2023-10-07T23:23:16.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c06b850 0x7f536c0cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 --2- 172.21.15.57:0/914042518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c158fb0 0x7f536c0ce6b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 --2- 172.21.15.57:0/914042518 >> v2:172.21.15.154:3300/0 conn(0x7f536c158be0 0x7f536c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 -- 172.21.15.57:0/914042518 >> 172.21.15.57:0/914042518 conn(0x7f536c068520 msgr2=0x7f536c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:16.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 -- 172.21.15.57:0/914042518 shutdown_connections 2023-10-07T23:23:16.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.277+0000 7f537083b700 1 -- 172.21.15.57:0/914042518 wait complete. 2023-10-07T23:23:16.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 Processor -- start 2023-10-07T23:23:16.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 -- start start 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c158be0 0x7f536c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 0x7f536c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c153130 con 0x7f536c06b850 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c16c940 con 0x7f536c158fb0 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f537083b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c16cc20 con 0x7f536c158be0 2023-10-07T23:23:16.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f536b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f536b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39152/0 (socket says 172.21.15.57:39152) 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f536b7fe700 1 -- 172.21.15.57:0/2151114871 learned_addr learned my addr 172.21.15.57:0/2151114871 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f5363fff700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c158be0 0x7f536c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.281+0000 7f536bfff700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 0x7f536c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 -- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c158be0 msgr2=0x7f536c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c158be0 0x7f536c15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 -- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 msgr2=0x7f536c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 0x7f536c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 -- 172.21.15.57:0/2151114871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53540094f0 con 0x7f536c06b850 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536bfff700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 0x7f536c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:23:16.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f536b7fe700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ec80 tx=0x7f535c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c00c9d0 con 0x7f536c06b850 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c00cb30 con 0x7f536c06b850 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c16cf00 con 0x7f536c06b850 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c013470 con 0x7f536c06b850 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c15d160 con 0x7f536c06b850 2023-10-07T23:23:16.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c15d3b0 con 0x7f536c06b850 2023-10-07T23:23:16.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.285+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f535c013820 con 0x7f536c06b850 2023-10-07T23:23:16.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.289+0000 7f53637fe700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 0x7f534c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.289+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f535c07c7c0 con 0x7f536c06b850 2023-10-07T23:23:16.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.289+0000 7f5363fff700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 0x7f534c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.289+0000 7f5363fff700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 0x7f534c062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f536c159430 tx=0x7f5354017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.293+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f536c15d3b0 con 0x7f536c06b850 2023-10-07T23:23:16.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f536c15d3b0 con 0x7f536c06b850 2023-10-07T23:23:16.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f53637fe700 1 -- 172.21.15.57:0/2151114871 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7f536c15d3b0 con 0x7f536c06b850 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: { 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:16.459 INFO:teuthology.orchestra.run.smithi057.stdout:] 2023-10-07T23:23:16.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 msgr2=0x7f534c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 0x7f534c062a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f536c159430 tx=0x7f5354017000 comp rx=0 tx=0).stop 2023-10-07T23:23:16.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 msgr2=0x7f536c13eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ec80 tx=0x7f535c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 shutdown_connections 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f534c060550 0x7f534c062a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f536c158fb0 0x7f536c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f536c158be0 0x7f536c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 --2- 172.21.15.57:0/2151114871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f536c06b850 0x7f536c13eac0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.457+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 >> 172.21.15.57:0/2151114871 conn(0x7f536c068520 msgr2=0x7f536c0d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.461+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 shutdown_connections 2023-10-07T23:23:16.462 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.461+0000 7f537083b700 1 -- 172.21.15.57:0/2151114871 wait complete. 2023-10-07T23:23:16.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-07T23:23:16.473 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-07T23:23:16.587 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 Processor -- start 2023-10-07T23:23:16.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 -- start start 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f572c0c8a50 0x7f572c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c0c75b0 con 0x7f572c0ae0f0 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c0c69e0 con 0x7f572c0add20 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f5730c78700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0c6c80 con 0x7f572c0c8a50 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f572affd700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f572c0ae0f0 0x7f572c156760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45778/0 (socket says 172.21.15.57:45778) 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f572affd700 1 -- 172.21.15.57:0/2426703229 learned_addr learned my addr 172.21.15.57:0/2426703229 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:16.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.585+0000 7f572bfff700 1 --2- 172.21.15.57:0/2426703229 >> v2:172.21.15.154:3300/0 conn(0x7f572c0c8a50 0x7f572c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572a7fc700 1 -- 172.21.15.57:0/2426703229 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247221893 0 0) 0x7f572c0c75b0 con 0x7f572c0ae0f0 2023-10-07T23:23:16.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572a7fc700 1 -- 172.21.15.57:0/2426703229 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5714003570 con 0x7f572c0ae0f0 2023-10-07T23:23:16.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572bfff700 1 -- 172.21.15.57:0/2426703229 >> v1:172.21.15.154:6789/0 conn(0x7f572c0add20 legacy=0x7f572c0abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572bfff700 1 -- 172.21.15.57:0/2426703229 >> v1:172.21.15.57:6789/0 conn(0x7f572c0ae0f0 legacy=0x7f572c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572bfff700 1 -- 172.21.15.57:0/2426703229 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c0c6f60 con 0x7f572c0c8a50 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572bfff700 1 --2- 172.21.15.57:0/2426703229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c156e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5724009e60 tx=0x7f5724009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ebb71efed98e30 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572a7fc700 1 -- 172.21.15.57:0/2426703229 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572400e070 con 0x7f572c0c8a50 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572a7fc700 1 -- 172.21.15.57:0/2426703229 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572400bb60 con 0x7f572c0c8a50 2023-10-07T23:23:16.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f572a7fc700 1 -- 172.21.15.57:0/2426703229 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5724012430 con 0x7f572c0c8a50 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 -- 172.21.15.57:0/2426703229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 msgr2=0x7f572c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 --2- 172.21.15.57:0/2426703229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c156e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5724009e60 tx=0x7f5724009780 comp rx=0 tx=0).stop 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 -- 172.21.15.57:0/2426703229 shutdown_connections 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 --2- 172.21.15.57:0/2426703229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c156e70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 -- 172.21.15.57:0/2426703229 >> 172.21.15.57:0/2426703229 conn(0x7f572c0670e0 msgr2=0x7f572c0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 -- 172.21.15.57:0/2426703229 shutdown_connections 2023-10-07T23:23:16.592 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 -- 172.21.15.57:0/2426703229 wait complete. 2023-10-07T23:23:16.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.589+0000 7f5730c78700 1 Processor -- start 2023-10-07T23:23:16.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- start start 2023-10-07T23:23:16.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f572c0add20 0x7f572c14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c0aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c14f430 con 0x7f572c0ae0f0 2023-10-07T23:23:16.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c16d430 con 0x7f572c0c8a50 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c9ac0 con 0x7f572c0add20 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44410/0 (socket says 172.21.15.57:44410) 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 -- 172.21.15.57:0/1522698742 learned_addr learned my addr 172.21.15.57:0/1522698742 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572b7fe700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f572c0add20 0x7f572c14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572bfff700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c0aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 -- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f572c0add20 msgr2=0x7f572c14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f572c0add20 0x7f572c14eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 -- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 msgr2=0x7f572c0aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:16.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c0aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:16.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 -- 172.21.15.57:0/1522698742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724009a20 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f572affd700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c7f0 tx=0x7f571c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c005cd0 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c005e50 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c16c540 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c007320 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.593+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c16cb00 con 0x7f572c0ae0f0 2023-10-07T23:23:16.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c0711c0 con 0x7f572c0ae0f0 2023-10-07T23:23:16.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f571c007690 con 0x7f572c0ae0f0 2023-10-07T23:23:16.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f57137fe700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 0x7f5718062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:16.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f571c07bad0 con 0x7f572c0ae0f0 2023-10-07T23:23:16.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f572b7fe700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 0x7f5718062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:16.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.597+0000 7f572b7fe700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 0x7f5718062a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f572c0684d0 tx=0x7f5720009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:16.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.605+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f571c04c410 con 0x7f572c0ae0f0 2023-10-07T23:23:16.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:16.765+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f572c0c7240 con 0x7f572c0ae0f0 2023-10-07T23:23:17.129 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.125+0000 7f57137fe700 1 -- 172.21.15.57:0/1522698742 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7f571c017020 con 0x7f572c0ae0f0 2023-10-07T23:23:17.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 msgr2=0x7f5718062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 0x7f5718062a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f572c0684d0 tx=0x7f5720009040 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 msgr2=0x7f572c0c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f571c00c7f0 tx=0x7f571c007e00 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 shutdown_connections 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5718060580 0x7f5718062a30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f572c0c8a50 0x7f572c0aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f572c0ae0f0 0x7f572c0c9580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 --2- 172.21.15.57:0/1522698742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f572c0add20 0x7f572c14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 >> 172.21.15.57:0/1522698742 conn(0x7f572c0670e0 msgr2=0x7f572c06a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:17.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 shutdown_connections 2023-10-07T23:23:17.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.133+0000 7f5730c78700 1 -- 172.21.15.57:0/1522698742 wait complete. 2023-10-07T23:23:17.138 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2023-10-07T23:23:17.150 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-07T23:23:17.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.261+0000 7fd5de36b700 1 Processor -- start 2023-10-07T23:23:17.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 -- start start 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5d8140790 0x7fd5d80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd5d80c7100 0x7fd5d806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d80cafc0 con 0x7fd5d81403c0 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80c8090 con 0x7fd5d8140790 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5de36b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80cc7d0 con 0x7fd5d80c7100 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5d8140790 0x7fd5d80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.268 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd5d8140790 0x7fd5d80cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44414/0 (socket says 172.21.15.57:44414) 2023-10-07T23:23:17.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 -- 172.21.15.57:0/1797348589 learned_addr learned my addr 172.21.15.57:0/1797348589 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:17.269 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5ddb6a700 1 --2- 172.21.15.57:0/1797348589 >> v2:172.21.15.154:3300/0 conn(0x7fd5d80c7100 0x7fd5d806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 -- 172.21.15.57:0/1797348589 >> v2:172.21.15.154:3300/0 conn(0x7fd5d80c7100 msgr2=0x7fd5d806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:17.270 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 --2- 172.21.15.57:0/1797348589 >> v2:172.21.15.154:3300/0 conn(0x7fd5d80c7100 0x7fd5d806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 -- 172.21.15.57:0/1797348589 >> v1:172.21.15.154:6789/0 conn(0x7fd5d81403c0 legacy=0x7fd5d80cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 -- 172.21.15.57:0/1797348589 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80ca690 con 0x7fd5d8140790 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.265+0000 7fd5dcb68700 1 --2- 172.21.15.57:0/1797348589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d80cb450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8069ea0 tx=0x7fd5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c4e75e65529de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5cffff700 1 -- 172.21.15.57:0/1797348589 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8009e70 con 0x7fd5d8140790 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5cffff700 1 -- 172.21.15.57:0/1797348589 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c800b860 con 0x7fd5d8140790 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5cffff700 1 -- 172.21.15.57:0/1797348589 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800bc00 con 0x7fd5d8140790 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- 172.21.15.57:0/1797348589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 msgr2=0x7fd5d80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- 172.21.15.57:0/1797348589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d80cb450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8069ea0 tx=0x7fd5c8009250 comp rx=0 tx=0).stop 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- 172.21.15.57:0/1797348589 shutdown_connections 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- 172.21.15.57:0/1797348589 >> v2:172.21.15.154:3300/0 conn(0x7fd5d80c7100 0x7fd5d806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- 172.21.15.57:0/1797348589 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d80cb450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- 172.21.15.57:0/1797348589 >> 172.21.15.57:0/1797348589 conn(0x7fd5d8095d30 msgr2=0x7fd5d806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:17.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- 172.21.15.57:0/1797348589 shutdown_connections 2023-10-07T23:23:17.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- 172.21.15.57:0/1797348589 wait complete. 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 Processor -- start 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- start start 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d80c7100 0x7fd5d80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d81403c0 0x7fd5d80c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80c5330 con 0x7fd5d8140790 2023-10-07T23:23:17.274 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80c1be0 con 0x7fd5d80c7100 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5de36b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80c2160 con 0x7fd5d81403c0 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44418/0 (socket says 172.21.15.57:44418) 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 -- 172.21.15.57:0/2772155431 learned_addr learned my addr 172.21.15.57:0/2772155431 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5dd369700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d80c7100 0x7fd5d80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5dcb68700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d81403c0 0x7fd5d80c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 -- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d81403c0 msgr2=0x7fd5d80c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d81403c0 0x7fd5d80c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 -- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d80c7100 msgr2=0x7fd5d80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d80c7100 0x7fd5d80c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:17.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.269+0000 7fd5ddb6a700 1 -- 172.21.15.57:0/2772155431 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80094f0 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5ddb6a700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009e90 tx=0x7fd5d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000cc20 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000f040 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d80c4af0 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0014480 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d80c50d0 con 0x7fd5d8140790 2023-10-07T23:23:17.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.273+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d80c22f0 con 0x7fd5d8140790 2023-10-07T23:23:17.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.277+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd5d00147f0 con 0x7fd5d8140790 2023-10-07T23:23:17.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.277+0000 7fd5cd7fa700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 0x7fd5bc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:17.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.277+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fd5d007d0a0 con 0x7fd5d8140790 2023-10-07T23:23:17.278 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.277+0000 7fd5dd369700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 0x7fd5bc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:17.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.277+0000 7fd5dd369700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 0x7fd5bc062a30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4003a80 tx=0x7fd5d4007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:17.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.285+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd5d0051300 con 0x7fd5d8140790 2023-10-07T23:23:17.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:17.445+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd5d80c1eb0 con 0x7fd5d8140790 2023-10-07T23:23:18.131 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.129+0000 7fd5cd7fa700 1 -- 172.21.15.57:0/2772155431 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd5d0050e90 con 0x7fd5d8140790 2023-10-07T23:23:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 msgr2=0x7fd5bc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 0x7fd5bc062a30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4003a80 tx=0x7fd5d4007ec0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 msgr2=0x7fd5d81594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009e90 tx=0x7fd5d000b040 comp rx=0 tx=0).stop 2023-10-07T23:23:18.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 shutdown_connections 2023-10-07T23:23:18.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd5d8140790 0x7fd5d81594e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd5d81403c0 0x7fd5d80c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd5bc060580 0x7fd5bc062a30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.145+0000 7fd5de36b700 1 --2- 172.21.15.57:0/2772155431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5d80c7100 0x7fd5d80c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.150 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.149+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 >> 172.21.15.57:0/2772155431 conn(0x7fd5d8095d30 msgr2=0x7fd5d813f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:18.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.149+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 shutdown_connections 2023-10-07T23:23:18.153 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.149+0000 7fd5de36b700 1 -- 172.21.15.57:0/2772155431 wait complete. 2023-10-07T23:23:18.155 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2023-10-07T23:23:18.168 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-07T23:23:18.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 Processor -- start 2023-10-07T23:23:18.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 -- start start 2023-10-07T23:23:18.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32b8159380 0x7f32b806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b806d1a0 con 0x7f32b806cd40 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b80caf60 con 0x7f32b8158fb0 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.277+0000 7f32be800700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b815db00 con 0x7f32b8159380 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32bcd9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f32b806cd40 0x7f32b815cdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45788/0 (socket says 172.21.15.57:45788) 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32bcd9d700 1 -- 172.21.15.57:0/4107594235 learned_addr learned my addr 172.21.15.57:0/4107594235 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:18.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b77fe700 1 --2- 172.21.15.57:0/4107594235 >> v2:172.21.15.154:3300/0 conn(0x7f32b8159380 0x7f32b806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3734121698 0 0) 0x7f32b0001490 con 0x7f32b806cd40 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003570 con 0x7f32b806cd40 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b77fe700 1 -- 172.21.15.57:0/4107594235 >> v1:172.21.15.154:6789/0 conn(0x7f32b8158fb0 legacy=0x7f32b80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b77fe700 1 -- 172.21.15.57:0/4107594235 >> v1:172.21.15.57:6789/0 conn(0x7f32b806cd40 legacy=0x7f32b815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671203012 0 0) 0x7f32a4003570 con 0x7f32b806cd40 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b77fe700 1 -- 172.21.15.57:0/4107594235 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80ca630 con 0x7f32b8159380 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b77fe700 1 --2- 172.21.15.57:0/4107594235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b806c800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32a00098e0 tx=0x7f32a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a700b42ba285034 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:18.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0001980 con 0x7f32b8159380 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a0011420 con 0x7f32b8159380 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32b6ffd700 1 -- 172.21.15.57:0/4107594235 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00117c0 con 0x7f32b8159380 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 -- 172.21.15.57:0/4107594235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 msgr2=0x7f32b806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 --2- 172.21.15.57:0/4107594235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b806c800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f32a00098e0 tx=0x7f32a00091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 -- 172.21.15.57:0/4107594235 shutdown_connections 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 --2- 172.21.15.57:0/4107594235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b806c800 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 -- 172.21.15.57:0/4107594235 >> 172.21.15.57:0/4107594235 conn(0x7f32b8068650 msgr2=0x7f32b8068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 -- 172.21.15.57:0/4107594235 shutdown_connections 2023-10-07T23:23:18.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.281+0000 7f32be800700 1 -- 172.21.15.57:0/4107594235 wait complete. 2023-10-07T23:23:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 Processor -- start 2023-10-07T23:23:18.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 -- start start 2023-10-07T23:23:18.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32b806cd40 0x7f32b8169480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81699c0 con 0x7f32b8158fb0 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b815a9a0 con 0x7f32b8159380 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32be800700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80ca3e0 con 0x7f32b806cd40 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32b77fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44430/0 (socket says 172.21.15.57:44430) 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32b77fe700 1 -- 172.21.15.57:0/2146611879 learned_addr learned my addr 172.21.15.57:0/2146611879 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32b7fff700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32b806cd40 0x7f32b8169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:18.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.285+0000 7f32bcd9d700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 -- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32b806cd40 msgr2=0x7f32b8169480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32b806cd40 0x7f32b8169480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 -- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 msgr2=0x7f32b8155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b8155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 -- 172.21.15.57:0/2146611879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009420 con 0x7f32b8158fb0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32b77fe700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f32b80cfdb0 tx=0x7f32a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a001caf0 con 0x7f32b8158fb0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a001cc70 con 0x7f32b8158fb0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80c9b40 con 0x7f32b8158fb0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a001a450 con 0x7f32b8158fb0 2023-10-07T23:23:18.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80ca100 con 0x7f32b8158fb0 2023-10-07T23:23:18.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.289+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b815ab30 con 0x7f32b8158fb0 2023-10-07T23:23:18.293 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.293+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f32a001a820 con 0x7f32b8158fb0 2023-10-07T23:23:18.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.293+0000 7f32affff700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 0x7f3298062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:18.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.293+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f32a007e250 con 0x7f32b8158fb0 2023-10-07T23:23:18.294 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.293+0000 7f32b7fff700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 0x7f3298062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:18.295 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.293+0000 7f32b7fff700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 0x7f3298062a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32b80cdfe0 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:18.303 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.301+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f32a004e8a0 con 0x7f32b8158fb0 2023-10-07T23:23:18.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:18.497+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f32b80cbcc0 con 0x7f32b8158fb0 2023-10-07T23:23:18.891 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:23:18.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:23:19.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.185+0000 7f32affff700 1 -- 172.21.15.57:0/2146611879 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v17) v1 ==== 172+0+0 (secure 0 0 0) 0x7f32a0051ec0 con 0x7f32b8158fb0 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 msgr2=0x7f3298062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 0x7f3298062a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32b80cdfe0 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 msgr2=0x7f32b80c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f32b80cfdb0 tx=0x7f32a0016000 comp rx=0 tx=0).stop 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 shutdown_connections 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3298060550 0x7f3298062a00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b8159380 0x7f32b8155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f32b8158fb0 0x7f32b80c9160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 --2- 172.21.15.57:0/2146611879 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f32b806cd40 0x7f32b8169480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 >> 172.21.15.57:0/2146611879 conn(0x7f32b8068650 msgr2=0x7f32b80ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.193 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.189+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 shutdown_connections 2023-10-07T23:23:19.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.197+0000 7f32be800700 1 -- 172.21.15.57:0/2146611879 wait complete. 2023-10-07T23:23:19.198 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 6 and data pool 7 2023-10-07T23:23:19.209 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:19.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.301+0000 7f6ce63f4700 1 Processor -- start 2023-10-07T23:23:19.305 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce63f4700 1 -- start start 2023-10-07T23:23:19.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce63f4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 0x7f6ce00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce63f4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00c9d40 con 0x7f6ce006ac90 2023-10-07T23:23:19.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce63f4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce006caa0 con 0x7f6ce00c9900 2023-10-07T23:23:19.307 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce63f4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00ad090 con 0x7f6ce0155db0 2023-10-07T23:23:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce4bf1700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6ce006ac90 0x7f6ce0156180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45796/0 (socket says 172.21.15.57:45796) 2023-10-07T23:23:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce4bf1700 1 -- 172.21.15.57:0/2065675131 learned_addr learned my addr 172.21.15.57:0/2065675131 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce53f2700 1 --2- 172.21.15.57:0/2065675131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 0x7f6ce00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce53f2700 1 -- 172.21.15.57:0/2065675131 >> v1:172.21.15.154:6789/0 conn(0x7f6ce00c9900 legacy=0x7f6ce0158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.308 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce53f2700 1 -- 172.21.15.57:0/2065675131 >> v1:172.21.15.57:6789/0 conn(0x7f6ce006ac90 legacy=0x7f6ce0156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce53f2700 1 -- 172.21.15.57:0/2065675131 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce006b1e0 con 0x7f6ce0155db0 2023-10-07T23:23:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6cd7fff700 1 -- 172.21.15.57:0/2065675131 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4399849 0 0) 0x7f6ce00c9d40 con 0x7f6ce006ac90 2023-10-07T23:23:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6ce53f2700 1 --2- 172.21.15.57:0/2065675131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 0x7f6ce00c93c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009910 tx=0x7f6cdc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2891958239c0f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6cd7fff700 1 -- 172.21.15.57:0/2065675131 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00b860 con 0x7f6ce0155db0 2023-10-07T23:23:19.309 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6cd7fff700 1 -- 172.21.15.57:0/2065675131 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00b9c0 con 0x7f6ce0155db0 2023-10-07T23:23:19.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.305+0000 7f6cd7fff700 1 -- 172.21.15.57:0/2065675131 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00bd60 con 0x7f6ce0155db0 2023-10-07T23:23:19.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2065675131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 msgr2=0x7f6ce00c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2065675131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 0x7f6ce00c93c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009910 tx=0x7f6cdc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2065675131 shutdown_connections 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2065675131 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce0155db0 0x7f6ce00c93c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2065675131 >> 172.21.15.57:0/2065675131 conn(0x7f6ce0067b80 msgr2=0x7f6ce00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2065675131 shutdown_connections 2023-10-07T23:23:19.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2065675131 wait complete. 2023-10-07T23:23:19.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.309+0000 7f6ce63f4700 1 Processor -- start 2023-10-07T23:23:19.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 -- start start 2023-10-07T23:23:19.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 0x7f6ce0168670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 0x7f6ce013f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0168bb0 con 0x7f6ce00c9900 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0140750 con 0x7f6ce0155db0 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce013feb0 con 0x7f6ce006ac90 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 0x7f6ce0168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce53f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce53f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:55314/0 (socket says 172.21.15.57:55314) 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce53f2700 1 -- 172.21.15.57:0/2305218793 learned_addr learned my addr 172.21.15.57:0/2305218793 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:19.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce5bf3700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 0x7f6ce013f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 -- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 msgr2=0x7f6ce0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 -- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 msgr2=0x7f6ce013f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 0x7f6ce013f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 -- 172.21.15.57:0/2305218793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc009420 con 0x7f6ce00c9900 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce5bf3700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 0x7f6ce013f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce53f2700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce4bf1700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 0x7f6ce0168670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c4e0 tx=0x7f6cd0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0005b30 con 0x7f6ce00c9900 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd0005cd0 con 0x7f6ce00c9900 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce0140190 con 0x7f6ce00c9900 2023-10-07T23:23:19.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.313+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0007320 con 0x7f6ce00c9900 2023-10-07T23:23:19.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce0167280 con 0x7f6ce00c9900 2023-10-07T23:23:19.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce01674a0 con 0x7f6ce00c9900 2023-10-07T23:23:19.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6cd0012400 con 0x7f6ce00c9900 2023-10-07T23:23:19.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6cd57fa700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 0x7f6cc4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6cd007bb50 con 0x7f6ce00c9900 2023-10-07T23:23:19.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.317+0000 7f6ce53f2700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 0x7f6cc4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.321+0000 7f6ce53f2700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 0x7f6cc4062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c010 tx=0x7f6cdc017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.325+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6cd004c150 con 0x7f6ce00c9900 2023-10-07T23:23:19.489 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.489+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce00711c0 con 0x7f6ce00c9900 2023-10-07T23:23:19.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.489+0000 7f6cd57fa700 1 -- 172.21.15.57:0/2305218793 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f6cd004f770 con 0x7f6ce00c9900 2023-10-07T23:23:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:19.492 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":46,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:23:18.502276+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 msgr2=0x7f6cc4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.494 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 0x7f6cc4062a00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c010 tx=0x7f6cdc017000 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 msgr2=0x7f6ce0168670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 0x7f6ce0168670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c4e0 tx=0x7f6cd0007f30 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 shutdown_connections 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6cc4060550 0x7f6cc4062a00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ce0155db0 0x7f6ce013f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ce00c9900 0x7f6ce0168670 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 --2- 172.21.15.57:0/2305218793 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6ce006ac90 0x7f6ce0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 >> 172.21.15.57:0/2305218793 conn(0x7f6ce0067b80 msgr2=0x7f6ce014f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.495 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 shutdown_connections 2023-10-07T23:23:19.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.493+0000 7f6ce63f4700 1 -- 172.21.15.57:0/2305218793 wait complete. 2023-10-07T23:23:19.506 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:19.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.613+0000 7fd82f59e700 1 Processor -- start 2023-10-07T23:23:19.616 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.613+0000 7fd82f59e700 1 -- start start 2023-10-07T23:23:19.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.613+0000 7fd82f59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd83006cf10 0x7fd8300cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300cafc0 con 0x7fd830140790 2023-10-07T23:23:19.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300cc7d0 con 0x7fd8301403c0 2023-10-07T23:23:19.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8300c8090 con 0x7fd83006cf10 2023-10-07T23:23:19.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82dd9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd830140790 0x7fd83006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45804/0 (socket says 172.21.15.57:45804) 2023-10-07T23:23:19.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82dd9b700 1 -- 172.21.15.57:0/3652007436 learned_addr learned my addr 172.21.15.57:0/3652007436 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:19.618 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82ed9d700 1 --2- 172.21.15.57:0/3652007436 >> v2:172.21.15.154:3300/0 conn(0x7fd83006cf10 0x7fd8300cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737822058 0 0) 0x7fd8300cafc0 con 0x7fd830140790 2023-10-07T23:23:19.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003570 con 0x7fd830140790 2023-10-07T23:23:19.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2578766960 0 0) 0x7fd81c003570 con 0x7fd830140790 2023-10-07T23:23:19.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82ed9d700 1 -- 172.21.15.57:0/3652007436 >> v1:172.21.15.154:6789/0 conn(0x7fd8301403c0 legacy=0x7fd8300cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82ed9d700 1 -- 172.21.15.57:0/3652007436 >> v1:172.21.15.57:6789/0 conn(0x7fd830140790 legacy=0x7fd83006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82ed9d700 1 -- 172.21.15.57:0/3652007436 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300ca690 con 0x7fd83006cf10 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82ed9d700 1 --2- 172.21.15.57:0/3652007436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd8300cb450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098e0 tx=0x7fd8180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f7b84310e8e9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280017f0 con 0x7fd83006cf10 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818011420 con 0x7fd83006cf10 2023-10-07T23:23:19.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82d59a700 1 -- 172.21.15.57:0/3652007436 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8180117a0 con 0x7fd83006cf10 2023-10-07T23:23:19.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- 172.21.15.57:0/3652007436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 msgr2=0x7fd8300cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 --2- 172.21.15.57:0/3652007436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd8300cb450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098e0 tx=0x7fd8180091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- 172.21.15.57:0/3652007436 shutdown_connections 2023-10-07T23:23:19.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 --2- 172.21.15.57:0/3652007436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd8300cb450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- 172.21.15.57:0/3652007436 >> 172.21.15.57:0/3652007436 conn(0x7fd830095d30 msgr2=0x7fd83006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- 172.21.15.57:0/3652007436 shutdown_connections 2023-10-07T23:23:19.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.617+0000 7fd82f59e700 1 -- 172.21.15.57:0/3652007436 wait complete. 2023-10-07T23:23:19.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 Processor -- start 2023-10-07T23:23:19.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 -- start start 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd83016a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd830140790 0x7fd830131b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830097100 con 0x7fd8301403c0 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830096840 con 0x7fd83006cf10 2023-10-07T23:23:19.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82f59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300af170 con 0x7fd830140790 2023-10-07T23:23:19.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44448/0 (socket says 172.21.15.57:44448) 2023-10-07T23:23:19.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82dd9b700 1 -- 172.21.15.57:0/676491498 learned_addr learned my addr 172.21.15.57:0/676491498 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:19.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82e59c700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd83016a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.626 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.621+0000 7fd82ed9d700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd830140790 0x7fd830131b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 -- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd830140790 msgr2=0x7fd830131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.627 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd830140790 0x7fd830131b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 -- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 msgr2=0x7fd83016a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd83016a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 -- 172.21.15.57:0/676491498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009420 con 0x7fd8301403c0 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82dd9b700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd82800cc40 tx=0x7fd828001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828004dc0 con 0x7fd8301403c0 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800a590 con 0x7fd8301403c0 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd830096ab0 con 0x7fd8301403c0 2023-10-07T23:23:19.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800a950 con 0x7fd8301403c0 2023-10-07T23:23:19.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300c54f0 con 0x7fd8301403c0 2023-10-07T23:23:19.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8300972b0 con 0x7fd8301403c0 2023-10-07T23:23:19.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.625+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd82800ad40 con 0x7fd8301403c0 2023-10-07T23:23:19.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.629+0000 7fd8267fc700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 0x7fd810062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.629+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd82807c190 con 0x7fd8301403c0 2023-10-07T23:23:19.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.629+0000 7fd82e59c700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 0x7fd810062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.629+0000 7fd82e59c700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 0x7fd810062a00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd820003a80 tx=0x7fd820007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.637+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd8300972b0 con 0x7fd8301403c0 2023-10-07T23:23:19.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.801+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8300972b0 con 0x7fd8301403c0 2023-10-07T23:23:19.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd8267fc700 1 -- 172.21.15.57:0/676491498 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fd8300972b0 con 0x7fd8301403c0 2023-10-07T23:23:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:19.806 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":46,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:23:18.502276+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 msgr2=0x7fd810062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 0x7fd810062a00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd820003a80 tx=0x7fd820007ec0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 msgr2=0x7fd83016aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.808 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd82800cc40 tx=0x7fd828001640 comp rx=0 tx=0).stop 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 shutdown_connections 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd830140790 0x7fd830131b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd8301403c0 0x7fd83016aaa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd810060550 0x7fd810062a00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 --2- 172.21.15.57:0/676491498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd83006cf10 0x7fd83016a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 >> 172.21.15.57:0/676491498 conn(0x7fd830095d30 msgr2=0x7fd8300ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 shutdown_connections 2023-10-07T23:23:19.809 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.805+0000 7fd82f59e700 1 -- 172.21.15.57:0/676491498 wait complete. 2023-10-07T23:23:19.821 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-07T23:23:19.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 Processor -- start 2023-10-07T23:23:19.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 -- start start 2023-10-07T23:23:19.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 0x7fdaa4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdaa40afd20 0x7fdaa406c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa415d9f0 con 0x7fdaa40afd20 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa406ac30 con 0x7fdaa4158c40 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaacba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40ca530 con 0x7fdaa415d580 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdaa40afd20 0x7fdaa406c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fdaa40afd20 0x7fdaa406c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44464/0 (socket says 172.21.15.57:44464) 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 -- 172.21.15.57:0/3356331404 learned_addr learned my addr 172.21.15.57:0/3356331404 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaa9cb8700 1 --2- 172.21.15.57:0/3356331404 >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 0x7fdaa4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaa94b7700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 -- 172.21.15.57:0/3356331404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 msgr2=0x7fdaa406a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:19.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa406a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 -- 172.21.15.57:0/3356331404 >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 msgr2=0x7fdaa4159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 --2- 172.21.15.57:0/3356331404 >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 0x7fdaa4159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaaa4b9700 1 -- 172.21.15.57:0/3356331404 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa406ae60 con 0x7fdaa40afd20 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.937+0000 7fdaa9cb8700 1 --2- 172.21.15.57:0/3356331404 >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 0x7fdaa4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaa4b9700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa406c800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40d0e20 tx=0x7fda9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4ca61a852aa639 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaa8cb6700 1 -- 172.21.15.57:0/3356331404 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00e070 con 0x7fdaa40afd20 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaa8cb6700 1 -- 172.21.15.57:0/3356331404 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c00b970 con 0x7fdaa40afd20 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaa8cb6700 1 -- 172.21.15.57:0/3356331404 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0092e0 con 0x7fdaa40afd20 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 -- 172.21.15.57:0/3356331404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 msgr2=0x7fdaa406c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa406c800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40d0e20 tx=0x7fda9c009840 comp rx=0 tx=0).stop 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 -- 172.21.15.57:0/3356331404 shutdown_connections 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa406c800 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 --2- 172.21.15.57:0/3356331404 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa406a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 --2- 172.21.15.57:0/3356331404 >> v2:172.21.15.154:3300/0 conn(0x7fdaa4158c40 0x7fdaa4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 -- 172.21.15.57:0/3356331404 >> 172.21.15.57:0/3356331404 conn(0x7fdaa4062ba0 msgr2=0x7fdaa4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 -- 172.21.15.57:0/3356331404 shutdown_connections 2023-10-07T23:23:19.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.941+0000 7fdaaacba700 1 -- 172.21.15.57:0/3356331404 wait complete. 2023-10-07T23:23:19.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 Processor -- start 2023-10-07T23:23:19.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- start start 2023-10-07T23:23:19.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4158c40 0x7fdaa41568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4153840 con 0x7fdaa40afd20 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40b3b60 con 0x7fdaa4158c40 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40b2c70 con 0x7fdaa415d580 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44468/0 (socket says 172.21.15.57:44468) 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 -- 172.21.15.57:0/765953764 learned_addr learned my addr 172.21.15.57:0/765953764 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:19.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa94b7700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4158c40 0x7fdaa41568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaa4b9700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 -- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 msgr2=0x7fdaa40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 -- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4158c40 msgr2=0x7fdaa41568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4158c40 0x7fdaa41568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 -- 172.21.15.57:0/765953764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009420 con 0x7fdaa40afd20 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaa9cb8700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000ec80 tx=0x7fdaa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000c920 con 0x7fdaa40afd20 2023-10-07T23:23:19.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa000ca80 con 0x7fdaa40afd20 2023-10-07T23:23:19.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c009ae0 con 0x7fdaa40afd20 2023-10-07T23:23:19.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000ce00 con 0x7fdaa40afd20 2023-10-07T23:23:19.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.945+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa40b3230 con 0x7fdaa40afd20 2023-10-07T23:23:19.954 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.949+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa40711c0 con 0x7fdaa40afd20 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.949+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fdaa0013680 con 0x7fdaa40afd20 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.949+0000 7fda99ffb700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 0x7fda88062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.949+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdaa007cba0 con 0x7fdaa40afd20 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.949+0000 7fdaa94b7700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 0x7fda88062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.953+0000 7fdaa94b7700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 0x7fda88062a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdaa41571e0 tx=0x7fda940091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:19.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:19.953+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fdaa004d0f0 con 0x7fdaa40afd20 2023-10-07T23:23:20.151 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.149+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdaa406b140 con 0x7fdaa40afd20 2023-10-07T23:23:20.178 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.177+0000 7fda99ffb700 1 -- 172.21.15.57:0/765953764 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdaa0050710 con 0x7fdaa40afd20 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 msgr2=0x7fda88062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 0x7fda88062a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdaa41571e0 tx=0x7fda940091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 msgr2=0x7fdaa4147f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000ec80 tx=0x7fdaa00091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 shutdown_connections 2023-10-07T23:23:20.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fdaa415d580 0x7fdaa40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fda88060580 0x7fda88062a30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdaa4158c40 0x7fdaa41568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 --2- 172.21.15.57:0/765953764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaa40afd20 0x7fdaa4147f90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 >> 172.21.15.57:0/765953764 conn(0x7fdaa4062ba0 msgr2=0x7fdaa4157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 shutdown_connections 2023-10-07T23:23:20.184 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.181+0000 7fdaaacba700 1 -- 172.21.15.57:0/765953764 wait complete. 2023-10-07T23:23:20.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-07T23:23:20.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.309+0000 7f53ceedc700 1 Processor -- start 2023-10-07T23:23:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.309+0000 7f53ceedc700 1 -- start start 2023-10-07T23:23:20.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53c80bed60 0x7f53c806b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53c80bf130 0x7f53c806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c81613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c806d1c0 con 0x7f53c80bed60 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c806ab50 con 0x7f53c80bf130 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c806b3f0 con 0x7f53c806cd40 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53c80bed60 0x7f53c806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f53c80bed60 0x7f53c806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44478/0 (socket says 172.21.15.57:44478) 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 -- 172.21.15.57:0/1804368844 learned_addr learned my addr 172.21.15.57:0/1804368844 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:20.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53bffff700 1 --2- 172.21.15.57:0/1804368844 >> v2:172.21.15.154:3300/0 conn(0x7f53c80bf130 0x7f53c806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53cd479700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c81613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 -- 172.21.15.57:0/1804368844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 msgr2=0x7f53c81613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:20.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c81613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 -- 172.21.15.57:0/1804368844 >> v2:172.21.15.154:3300/0 conn(0x7f53c80bf130 msgr2=0x7f53c806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 --2- 172.21.15.57:0/1804368844 >> v2:172.21.15.154:3300/0 conn(0x7f53c80bf130 0x7f53c806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 -- 172.21.15.57:0/1804368844 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c815dc70 con 0x7f53c80bed60 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ccc78700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bed60 0x7f53c806b880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d850 tx=0x7f53c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee54d87acf514b29 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53bf7fe700 1 -- 172.21.15.57:0/1804368844 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400b640 con 0x7f53c80bed60 2023-10-07T23:23:20.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53bf7fe700 1 -- 172.21.15.57:0/1804368844 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400b7a0 con 0x7f53c80bed60 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53bf7fe700 1 -- 172.21.15.57:0/1804368844 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400bb60 con 0x7f53c80bed60 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- 172.21.15.57:0/1804368844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bed60 msgr2=0x7f53c806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bed60 0x7f53c806b880 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d850 tx=0x7f53c4009250 comp rx=0 tx=0).stop 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- 172.21.15.57:0/1804368844 shutdown_connections 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c81613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- 172.21.15.57:0/1804368844 >> v2:172.21.15.154:3300/0 conn(0x7f53c80bf130 0x7f53c806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 --2- 172.21.15.57:0/1804368844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bed60 0x7f53c806b880 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- 172.21.15.57:0/1804368844 >> 172.21.15.57:0/1804368844 conn(0x7f53c8062ba0 msgr2=0x7f53c8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:20.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- 172.21.15.57:0/1804368844 shutdown_connections 2023-10-07T23:23:20.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.313+0000 7f53ceedc700 1 -- 172.21.15.57:0/1804368844 wait complete. 2023-10-07T23:23:20.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 Processor -- start 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- start start 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80bed60 0x7f53c80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c1500 con 0x7f53c80bf130 2023-10-07T23:23:20.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815bb60 con 0x7f53c80bed60 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80c0610 con 0x7f53c806cd40 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44492/0 (socket says 172.21.15.57:44492) 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 -- 172.21.15.57:0/4142177591 learned_addr learned my addr 172.21.15.57:0/4142177591 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ccc78700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53bffff700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80bed60 0x7f53c80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 -- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 msgr2=0x7f53c815b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c815b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 -- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80bed60 msgr2=0x7f53c80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80bed60 0x7f53c80b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 -- 172.21.15.57:0/4142177591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40094f0 con 0x7f53c80bf130 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53cd479700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009e90 tx=0x7f53c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000cb70 con 0x7f53c80bf130 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c000f040 con 0x7f53c80bf130 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80c08f0 con 0x7f53c80bf130 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0014480 con 0x7f53c80bf130 2023-10-07T23:23:20.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.317+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c0eb0 con 0x7f53c80bf130 2023-10-07T23:23:20.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80c8f00 con 0x7f53c80bf130 2023-10-07T23:23:20.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f53c0014830 con 0x7f53c80bf130 2023-10-07T23:23:20.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53bcff9700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 0x7f53b4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:20.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53c007d120 con 0x7f53c80bf130 2023-10-07T23:23:20.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53ccc78700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 0x7f53b4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:20.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.321+0000 7f53ccc78700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 0x7f53b4062a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d850 tx=0x7f53c4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:20.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.333+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f53c80c8f00 con 0x7f53c80bf130 2023-10-07T23:23:20.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:20.533+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f53c80c8f00 con 0x7f53c80bf130 2023-10-07T23:23:21.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.205+0000 7f53bcff9700 1 -- 172.21.15.57:0/4142177591 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53c80c8f00 con 0x7f53c80bf130 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 msgr2=0x7f53b4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 0x7f53b4062a00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d850 tx=0x7f53c4017000 comp rx=0 tx=0).stop 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 msgr2=0x7f53c815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009e90 tx=0x7f53c000b040 comp rx=0 tx=0).stop 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 shutdown_connections 2023-10-07T23:23:21.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53c80bf130 0x7f53c815fe60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f53b4060550 0x7f53b4062a00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80bed60 0x7f53c80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 --2- 172.21.15.57:0/4142177591 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f53c806cd40 0x7f53c815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 >> 172.21.15.57:0/4142177591 conn(0x7f53c8062ba0 msgr2=0x7f53c815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 shutdown_connections 2023-10-07T23:23:21.213 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.209+0000 7f53ceedc700 1 -- 172.21.15.57:0/4142177591 wait complete. 2023-10-07T23:23:21.229 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:21.346 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 Processor -- start 2023-10-07T23:23:21.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 -- start start 2023-10-07T23:23:21.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ab00cba50 0x7f2ab00cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ab00c6c10 0x7f2ab00c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00cac40 con 0x7f2ab00bee60 2023-10-07T23:23:21.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00cc3c0 con 0x7f2ab00cba50 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab63ca700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00ca030 con 0x7f2ab00c6c10 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ab00cba50 0x7f2ab00cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2ab00cba50 0x7f2ab00cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44504/0 (socket says 172.21.15.57:44504) 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3721262605 learned_addr learned my addr 172.21.15.57:0/3721262605 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2ab4967700 1 --2- 172.21.15.57:0/3721262605 >> v2:172.21.15.154:3300/0 conn(0x7f2ab00c6c10 0x7f2ab00c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3721262605 >> v2:172.21.15.154:3300/0 conn(0x7f2ab00c6c10 msgr2=0x7f2ab00c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 --2- 172.21.15.57:0/3721262605 >> v2:172.21.15.154:3300/0 conn(0x7f2ab00c6c10 0x7f2ab00c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3721262605 >> v1:172.21.15.154:6789/0 conn(0x7f2ab00bee60 legacy=0x7f2ab00cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3721262605 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00ca310 con 0x7f2ab00cba50 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaeffd700 1 -- 172.21.15.57:0/3721262605 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212875589 0 0) 0x7f2ab00cac40 con 0x7f2ab00bee60 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaf7fe700 1 --2- 172.21.15.57:0/3721262605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00cba50 0x7f2ab00cbe80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00ae3f0 tx=0x7f2aa8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68679d0fb3540851 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaeffd700 1 -- 172.21.15.57:0/3721262605 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800e070 con 0x7f2ab00cba50 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaeffd700 1 -- 172.21.15.57:0/3721262605 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa800ba20 con 0x7f2ab00cba50 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.344+0000 7f2aaeffd700 1 -- 172.21.15.57:0/3721262605 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80092e0 con 0x7f2ab00cba50 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3721262605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00cba50 msgr2=0x7f2ab00cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3721262605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00cba50 0x7f2ab00cbe80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00ae3f0 tx=0x7f2aa8009840 comp rx=0 tx=0).stop 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3721262605 shutdown_connections 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3721262605 >> v2:172.21.15.154:3300/0 conn(0x7f2ab00c6c10 0x7f2ab00c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3721262605 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00cba50 0x7f2ab00cbe80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3721262605 >> 172.21.15.57:0/3721262605 conn(0x7f2ab00670e0 msgr2=0x7f2ab00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3721262605 shutdown_connections 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3721262605 wait complete. 2023-10-07T23:23:21.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 Processor -- start 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- start start 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ab00bee60 0x7f2ab00b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ab00cba50 0x7f2ab0155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00af880 con 0x7f2ab00c6c10 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00afb60 con 0x7f2ab00bee60 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2ab63ca700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00afe40 con 0x7f2ab00cba50 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.348+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2ab4967700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ab00cba50 0x7f2ab0155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44520/0 (socket says 172.21.15.57:44520) 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ab00bee60 0x7f2ab00b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3341677412 learned_addr learned my addr 172.21.15.57:0/3341677412 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ab00cba50 msgr2=0x7f2ab0155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ab00cba50 0x7f2ab0155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ab00bee60 msgr2=0x7f2ab00b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ab00bee60 0x7f2ab00b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 -- 172.21.15.57:0/3341677412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8009ae0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aaf7fe700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009ab0 tx=0x7f2aa8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80078f0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8007a70 con 0x7f2ab00c6c10 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab00b0120 con 0x7f2ab00c6c10 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8007e30 con 0x7f2ab00c6c10 2023-10-07T23:23:21.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab00b13e0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab00b15a0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.352+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2aa8003040 con 0x7f2ab00c6c10 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.356+0000 7f2aa7fff700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 0x7f2a90062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.356+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2aa807d990 con 0x7f2ab00c6c10 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.356+0000 7f2aaffff700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 0x7f2a90062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.356+0000 7f2aaffff700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 0x7f2a90062a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ab006b110 tx=0x7f2a98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.360+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2aa804df80 con 0x7f2ab00c6c10 2023-10-07T23:23:21.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.564+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab00711c0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.564+0000 7f2aa7fff700 1 -- 172.21.15.57:0/3341677412 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f2aa80515a0 con 0x7f2ab00c6c10 2023-10-07T23:23:21.568 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:21.568 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:23:21.186753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:23:21.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 msgr2=0x7f2a90062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 0x7f2a90062a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ab006b110 tx=0x7f2a98009040 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 msgr2=0x7f2ab0154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009ab0 tx=0x7f2aa8018000 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 shutdown_connections 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2ab00cba50 0x7f2ab0155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2ab00c6c10 0x7f2ab0154b50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2a90060580 0x7f2a90062a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 --2- 172.21.15.57:0/3341677412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ab00bee60 0x7f2ab00b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 >> 172.21.15.57:0/3341677412 conn(0x7f2ab00670e0 msgr2=0x7f2ab015b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 shutdown_connections 2023-10-07T23:23:21.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.568+0000 7f2ab63ca700 1 -- 172.21.15.57:0/3341677412 wait complete. 2023-10-07T23:23:21.572 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 19 2023-10-07T23:23:21.582 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:23:21.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 Processor -- start 2023-10-07T23:23:21.686 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- start start 2023-10-07T23:23:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00bee60 0x7f07b00cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b00cba50 0x7f07b00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b00cac40 con 0x7f07b00c6c10 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00cc3e0 con 0x7f07b00cba50 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00c7780 con 0x7f07b00bee60 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b00cba50 0x7f07b00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b00cba50 0x7f07b00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44526/0 (socket says 172.21.15.57:44526) 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 -- 172.21.15.57:0/1180677019 learned_addr learned my addr 172.21.15.57:0/1180677019 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07affff700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00bee60 0x7f07b00cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 -- 172.21.15.57:0/1180677019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00bee60 msgr2=0x7f07b00cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00bee60 0x7f07b00cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 -- 172.21.15.57:0/1180677019 >> v1:172.21.15.57:6789/0 conn(0x7f07b00c6c10 legacy=0x7f07b00c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.689 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 -- 172.21.15.57:0/1180677019 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00cb610 con 0x7f07b00cba50 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07af7fe700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00cbea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009f20 tx=0x7f07a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3718fb1435b4cd3e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07aeffd700 1 -- 172.21.15.57:0/1180677019 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000e070 con 0x7f07b00cba50 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07aeffd700 1 -- 172.21.15.57:0/1180677019 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a000ba80 con 0x7f07b00cba50 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07aeffd700 1 -- 172.21.15.57:0/1180677019 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00092e0 con 0x7f07b00cba50 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- 172.21.15.57:0/1180677019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 msgr2=0x7f07b00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00cbea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009f20 tx=0x7f07a0009840 comp rx=0 tx=0).stop 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- 172.21.15.57:0/1180677019 shutdown_connections 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00cbea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 --2- 172.21.15.57:0/1180677019 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00bee60 0x7f07b00cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- 172.21.15.57:0/1180677019 >> 172.21.15.57:0/1180677019 conn(0x7f07b00670e0 msgr2=0x7f07b00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- 172.21.15.57:0/1180677019 shutdown_connections 2023-10-07T23:23:21.690 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.684+0000 7f07b655c700 1 -- 172.21.15.57:0/1180677019 wait complete. 2023-10-07T23:23:21.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 Processor -- start 2023-10-07T23:23:21.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- start start 2023-10-07T23:23:21.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b00bee60 0x7f07b015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 0x7f07b00c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b015ee90 con 0x7f07b00cba50 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b015e280 con 0x7f07b00bee60 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b015e560 con 0x7f07b00c6c10 2023-10-07T23:23:21.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44532/0 (socket says 172.21.15.57:44532) 2023-10-07T23:23:21.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 -- 172.21.15.57:0/129985283 learned_addr learned my addr 172.21.15.57:0/129985283 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:21.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07af7fe700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 0x7f07b00c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07affff700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b00bee60 0x7f07b015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.699 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 -- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 msgr2=0x7f07b00c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 0x7f07b00c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 -- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b00bee60 msgr2=0x7f07b015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b00bee60 0x7f07b015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 -- 172.21.15.57:0/129985283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0009ae0 con 0x7f07b00cba50 2023-10-07T23:23:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07af7fe700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 0x7f07b00c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b4af9700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c560 tx=0x7f07a8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011590 con 0x7f07b00cba50 2023-10-07T23:23:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80116f0 con 0x7f07b00cba50 2023-10-07T23:23:21.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8011ab0 con 0x7f07b00cba50 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b015e840 con 0x7f07b00cba50 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.688+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b015f860 con 0x7f07b00cba50 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b00aa730 con 0x7f07b00cba50 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f07a801b3b0 con 0x7f07b00cba50 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07a7fff700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 0x7f0794062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f07a807bda0 con 0x7f07b00cba50 2023-10-07T23:23:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07affff700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 0x7f0794062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.692+0000 7f07affff700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 0x7f0794062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.696+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f07b00aa730 con 0x7f07b00cba50 2023-10-07T23:23:21.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.860+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07b00aa730 con 0x7f07b00cba50 2023-10-07T23:23:21.865 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.864+0000 7f07a7fff700 1 -- 172.21.15.57:0/129985283 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f07b00aa730 con 0x7f07b00cba50 2023-10-07T23:23:21.866 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:21.866 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":47,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:23:20.173418+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:23:21.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.864+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 msgr2=0x7f0794062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.864+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 0x7f0794062a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0798006fd0 tx=0x7f0798008040 comp rx=0 tx=0).stop 2023-10-07T23:23:21.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.864+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 msgr2=0x7f07b00bf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.864+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c560 tx=0x7f07a8007ef0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 shutdown_connections 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b00cba50 0x7f07b00bf910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b00c6c10 0x7f07b00c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0794060550 0x7f0794062a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 --2- 172.21.15.57:0/129985283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b00bee60 0x7f07b015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 >> 172.21.15.57:0/129985283 conn(0x7f07b00670e0 msgr2=0x7f07b006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:21.871 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 shutdown_connections 2023-10-07T23:23:21.872 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.868+0000 7f07b655c700 1 -- 172.21.15.57:0/129985283 wait complete. 2023-10-07T23:23:21.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-07T23:23:21.887 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:23:21.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 Processor -- start 2023-10-07T23:23:21.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 -- start start 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e4159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f18e40cae70 0x7f18e4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e415dac0 con 0x7f18e4159fd0 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e415cf40 con 0x7f18e40cae70 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18ec4e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e415d220 con 0x7f18e4158c90 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18e9a7f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f18e4159fd0 0x7f18e40ca760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45822/0 (socket says 172.21.15.57:45822) 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18e9a7f700 1 -- 172.21.15.57:0/3099451062 learned_addr learned my addr 172.21.15.57:0/3099451062 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:21.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.992+0000 7f18eaa81700 1 --2- 172.21.15.57:0/3099451062 >> v2:172.21.15.57:3300/0 conn(0x7f18e40cae70 0x7f18e4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ea280700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e4159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18eaa81700 1 -- 172.21.15.57:0/3099451062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 msgr2=0x7f18e4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18eaa81700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e4159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18eaa81700 1 -- 172.21.15.57:0/3099451062 >> v1:172.21.15.57:6789/0 conn(0x7f18e4159fd0 legacy=0x7f18e40ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18eaa81700 1 -- 172.21.15.57:0/3099451062 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e415d500 con 0x7f18e40cae70 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18eaa81700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e40cae70 0x7f18e4068670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009e60 tx=0x7f18dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe804553b491dd79 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18e927e700 1 -- 172.21.15.57:0/3099451062 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00e070 con 0x7f18e40cae70 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18e927e700 1 -- 172.21.15.57:0/3099451062 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc00bc30 con 0x7f18e40cae70 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18e927e700 1 -- 172.21.15.57:0/3099451062 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc012430 con 0x7f18e40cae70 2023-10-07T23:23:21.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- 172.21.15.57:0/3099451062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e40cae70 msgr2=0x7f18e4068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e40cae70 0x7f18e4068670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009e60 tx=0x7f18dc009780 comp rx=0 tx=0).stop 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- 172.21.15.57:0/3099451062 shutdown_connections 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e40cae70 0x7f18e4068670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/3099451062 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e4159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- 172.21.15.57:0/3099451062 >> 172.21.15.57:0/3099451062 conn(0x7f18e40cce30 msgr2=0x7f18e40c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- 172.21.15.57:0/3099451062 shutdown_connections 2023-10-07T23:23:22.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- 172.21.15.57:0/3099451062 wait complete. 2023-10-07T23:23:22.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 Processor -- start 2023-10-07T23:23:22.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:21.996+0000 7f18ec4e4700 1 -- start start 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e40cae70 0x7f18e40c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40af100 con 0x7f18e4159fd0 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40af3e0 con 0x7f18e40cae70 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40c4600 con 0x7f18e4158c90 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44556/0 (socket says 172.21.15.57:44556) 2023-10-07T23:23:22.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 -- 172.21.15.57:0/1614421082 learned_addr learned my addr 172.21.15.57:0/1614421082 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ea280700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e40cae70 0x7f18e40c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18e9a7f700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 -- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 msgr2=0x7f18e41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 -- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e40cae70 msgr2=0x7f18e40c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e40cae70 0x7f18e40c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 -- 172.21.15.57:0/1614421082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009a20 con 0x7f18e4159fd0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18eaa81700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f18e40c31b0 tx=0x7f18dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc012e80 con 0x7f18e4159fd0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e40c48e0 con 0x7f18e4159fd0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc027470 con 0x7f18e4159fd0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e41566e0 con 0x7f18e4159fd0 2023-10-07T23:23:22.006 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.000+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0277f0 con 0x7f18e4159fd0 2023-10-07T23:23:22.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e40711c0 con 0x7f18e4159fd0 2023-10-07T23:23:22.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f18dc024070 con 0x7f18e4159fd0 2023-10-07T23:23:22.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18da7fc700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 0x7f18c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18dc07eec0 con 0x7f18e4159fd0 2023-10-07T23:23:22.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18ea280700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 0x7f18c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.004+0000 7f18ea280700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 0x7f18c8062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006fd0 tx=0x7f18e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.012+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f18dc04f4b0 con 0x7f18e4159fd0 2023-10-07T23:23:22.171 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.168+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18e40af520 con 0x7f18c8060550 2023-10-07T23:23:22.188 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.184+0000 7f18da7fc700 1 -- 172.21.15.57:0/1614421082 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272613 (secure 0 0 0) 0x7f18e40af520 con 0x7f18c8060550 2023-10-07T23:23:22.192 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:22.194 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 msgr2=0x7f18c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 0x7f18c8062a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006fd0 tx=0x7f18e0008040 comp rx=0 tx=0).stop 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 msgr2=0x7f18e40c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f18e40c31b0 tx=0x7f18dc019000 comp rx=0 tx=0).stop 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 shutdown_connections 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f18e4159fd0 0x7f18e40c3dd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f18e4158c90 0x7f18e41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f18c8060550 0x7f18c8062a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.195 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 --2- 172.21.15.57:0/1614421082 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18e40cae70 0x7f18e40c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 >> 172.21.15.57:0/1614421082 conn(0x7f18e40cce30 msgr2=0x7f18e4156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 shutdown_connections 2023-10-07T23:23:22.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.192+0000 7f18ec4e4700 1 -- 172.21.15.57:0/1614421082 wait complete. 2023-10-07T23:23:22.196 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:23:22.212 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-07T23:23:21.110681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2399720,"kb_used_data":5032,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575000,"statfs":{"total":773094113280,"available":770636800000,"internally_reserved":0,"allocated":5152768,"data_stored":2718704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001284"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-07T23:23:14.097236+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:23:14.097236+0000","last_peered":"2023-10-07T23:23:14.097236+0000","last_clean":"2023-10-07T23:23:14.097236+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:23:14.097236+0000","last_undegraded":"2023-10-07T23:23:14.097236+0000","last_fullsized":"2023-10-07T23:23:14.097236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073488,"statfs":{"total":96636764160,"available":96331251712,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95199999999999996,"15min":0.94999999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.216,"5min":1.258,"15min":1.258},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3149999999999999,"5min":1.508,"15min":1.508},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.872,"15min":0.872},"min":{"1min":0.67100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.04,"5min":1.302,"15min":1.302},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94399999999999995,"15min":0.95299999999999996},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3660000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.046},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.327,"5min":1.365,"15min":1.365},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.0720000000000001,"15min":1.075},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.341,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1619999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2549999999999999,"5min":1.51,"15min":1.51},"last":0.872}]},{"osd":4,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.385,"5min":1.603,"15min":1.603},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.107,"15min":1.127},"min":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2809999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.194,"15min":1.216},"min":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.905,"15min":1.905},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.2270000000000001},"min":{"1min":0.85299999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.341,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.232,"15min":1.24},"min":{"1min":0.91300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6160000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.96199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.054},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.115},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.89600000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.196}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.157}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.236}]}]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0489999999999999,"15min":1.052},"min":{"1min":0.89100000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2909999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":0.97699999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0600000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88,"15min":0.878},"min":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1299999999999999,"5min":1.296,"15min":1.296},"last":0.78400000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.94399999999999995,"15min":0.94099999999999995},"min":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.222,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2549999999999999,"5min":2.452,"15min":2.452},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.008,"15min":1.024},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2050000000000001,"5min":2.649,"15min":2.649},"last":1.087},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2050000000000001,"5min":2.077,"15min":2.077},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.95799999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2350000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.1220000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2509999999999999,"5min":2.395,"15min":2.395},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.149,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.93700000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.286,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.198,"15min":1.21},"min":{"1min":0.82099999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6100000000000001,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.292,"15min":1.3160000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.3080000000000001,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.93300000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.671},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3720000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.034},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.2769999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.5740000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.778},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.645}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.706},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":2.1920000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.752},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.47}]}]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.81699999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.625},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.88900000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.0049999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.171},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.052}]},{"osd":7,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.323}]}]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57299999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.266,"15min":1.266},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.501,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.546,"15min":1.546},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90800000000000003,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.288,"5min":1.49,"15min":1.49},"last":0.44700000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0449999999999999,"15min":1.046},"min":{"1min":0.77700000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.1020000000000001,"15min":1.1120000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.395,"5min":1.871,"15min":1.871},"last":0.63800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.123,"15min":1.141},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.282,"5min":2.302,"15min":2.302},"last":0.876},{"interface":"front","average":{"1min":1.125,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7190000000000001,"5min":1.78,"15min":1.78},"last":0.92200000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2370000000000001,"15min":1.2390000000000001},"min":{"1min":0.94499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4790000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.048},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.452,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.2909999999999999},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4390000000000001,"5min":2.282,"15min":2.282},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.339,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.98799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302448,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.139,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.873}]},{"osd":2,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.59499999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0549999999999999,"5min":1.333,"15min":1.333},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1779999999999999,"5min":1.252,"15min":1.252},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.546,"15min":1.546},"last":0.90500000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4119999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.006},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.087,"15min":1.093},"min":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.93300000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1739999999999999,"15min":1.1759999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.04},{"interface":"front","average":{"1min":1.123,"5min":1.206,"15min":1.22},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4339999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.573,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2669999999999999,"15min":1.2729999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5249999999999999,"5min":2.093,"15min":2.093},"last":1.256}]},{"osd":7,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.4930000000000001,"5min":2.524,"15min":2.524},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.91800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6519999999999999,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.228}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.59599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76800000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92100000000000004,"15min":0.91200000000000003},"min":{"1min":0.77100000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.746},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.96599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.88700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.343,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2}]},{"osd":5,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1000000000000001,"15min":1.101},"min":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.421,"5min":1.51,"15min":1.51},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.175,"15min":1.169},"min":{"1min":0.91700000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7669999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2070000000000001,"15min":1.2010000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.325},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.577,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.081}]},{"osd":7,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.286,"15min":1.276},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.238}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:23:22.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:23:22.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 Processor -- start 2023-10-07T23:23:22.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 -- start start 2023-10-07T23:23:22.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c15d0e0 con 0x7f7c7c158f30 2023-10-07T23:23:22.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c15cad0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c80dd2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0b03d0 con 0x7f7c7c158b60 2023-10-07T23:23:22.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44568/0 (socket says 172.21.15.57:44568) 2023-10-07T23:23:22.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 -- 172.21.15.57:0/3207099346 learned_addr learned my addr 172.21.15.57:0/3207099346 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:22.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7b7fe700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 -- 172.21.15.57:0/3207099346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 msgr2=0x7f7c7c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 -- 172.21.15.57:0/3207099346 >> v1:172.21.15.154:6789/0 conn(0x7f7c7c158f30 legacy=0x7f7c7c06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.328+0000 7f7c7affd700 1 -- 172.21.15.57:0/3207099346 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c15cd70 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c7affd700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009f20 tx=0x7f7c64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe32b523ed5d9f49 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c7a7fc700 1 -- 172.21.15.57:0/3207099346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400e070 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c7a7fc700 1 -- 172.21.15.57:0/3207099346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6400b9d0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c7a7fc700 1 -- 172.21.15.57:0/3207099346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640092e0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 -- 172.21.15.57:0/3207099346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 msgr2=0x7f7c7c0afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009f20 tx=0x7f7c64009840 comp rx=0 tx=0).stop 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 -- 172.21.15.57:0/3207099346 shutdown_connections 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0afb40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/3207099346 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 -- 172.21.15.57:0/3207099346 >> 172.21.15.57:0/3207099346 conn(0x7f7c7c06b730 msgr2=0x7f7c7c0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 -- 172.21.15.57:0/3207099346 shutdown_connections 2023-10-07T23:23:22.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.332+0000 7f7c80dd2700 1 -- 172.21.15.57:0/3207099346 wait complete. 2023-10-07T23:23:22.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 Processor -- start 2023-10-07T23:23:22.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- start start 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c7c158f30 0x7f7c7c168b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cabd0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cb660 con 0x7f7c7c158f30 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0c9e00 con 0x7f7c7c158b60 2023-10-07T23:23:22.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44580/0 (socket says 172.21.15.57:44580) 2023-10-07T23:23:22.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 -- 172.21.15.57:0/2816919832 learned_addr learned my addr 172.21.15.57:0/2816919832 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:22.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7affd700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7bfff700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c7c158f30 0x7f7c7c168b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 -- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 msgr2=0x7f7c7c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 -- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c7c158f30 msgr2=0x7f7c7c168b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c7c158f30 0x7f7c7c168b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 -- 172.21.15.57:0/2816919832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c64009ae0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c7b7fe700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ec80 tx=0x7f7c6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00c850 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00c9b0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c0cadc0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00cd50 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.336+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c0cb380 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.342 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7c6c013680 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7c0711c0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c737fe700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 0x7f7c5c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c6c07ca70 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.343 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c7affd700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 0x7f7c5c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.340+0000 7f7c7affd700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 0x7f7c5c062a30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009880 tx=0x7f7c64018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.349 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.348+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7c6c04cfc0 con 0x7f7c7c0b1e80 2023-10-07T23:23:22.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.504+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c7c0b06b0 con 0x7f7c5c060580 2023-10-07T23:23:22.526 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.524+0000 7f7c737fe700 1 -- 172.21.15.57:0/2816919832 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272613 (secure 0 0 0) 0x7f7c7c0b06b0 con 0x7f7c5c060580 2023-10-07T23:23:22.529 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 msgr2=0x7f7c5c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 0x7f7c5c062a30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009880 tx=0x7f7c64018000 comp rx=0 tx=0).stop 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 msgr2=0x7f7c7c0c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ec80 tx=0x7f7c6c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 shutdown_connections 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c5c060580 0x7f7c5c062a30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c7c158f30 0x7f7c7c168b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c7c158b60 0x7f7c7c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 --2- 172.21.15.57:0/2816919832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c7c0b1e80 0x7f7c7c0c1bc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 >> 172.21.15.57:0/2816919832 conn(0x7f7c7c06b730 msgr2=0x7f7c7c0ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 shutdown_connections 2023-10-07T23:23:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.528+0000 7f7c80dd2700 1 -- 172.21.15.57:0/2816919832 wait complete. 2023-10-07T23:23:22.533 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:23:22.547 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-07T23:23:21.110681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2399720,"kb_used_data":5032,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575000,"statfs":{"total":773094113280,"available":770636800000,"internally_reserved":0,"allocated":5152768,"data_stored":2718704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001284"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-07T23:23:14.097236+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:23:14.097236+0000","last_peered":"2023-10-07T23:23:14.097236+0000","last_clean":"2023-10-07T23:23:14.097236+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:23:14.097236+0000","last_undegraded":"2023-10-07T23:23:14.097236+0000","last_fullsized":"2023-10-07T23:23:14.097236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073488,"statfs":{"total":96636764160,"available":96331251712,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95199999999999996,"15min":0.94999999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.216,"5min":1.258,"15min":1.258},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3149999999999999,"5min":1.508,"15min":1.508},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.872,"15min":0.872},"min":{"1min":0.67100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.04,"5min":1.302,"15min":1.302},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94399999999999995,"15min":0.95299999999999996},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3660000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.046},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.327,"5min":1.365,"15min":1.365},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.0720000000000001,"15min":1.075},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.341,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1619999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2549999999999999,"5min":1.51,"15min":1.51},"last":0.872}]},{"osd":4,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.385,"5min":1.603,"15min":1.603},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.107,"15min":1.127},"min":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2809999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.194,"15min":1.216},"min":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.905,"15min":1.905},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.2270000000000001},"min":{"1min":0.85299999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.341,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.232,"15min":1.24},"min":{"1min":0.91300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6160000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.96199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.054},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.115},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.89600000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.196}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.157}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.236}]}]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0489999999999999,"15min":1.052},"min":{"1min":0.89100000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2909999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":0.97699999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0600000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88,"15min":0.878},"min":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1299999999999999,"5min":1.296,"15min":1.296},"last":0.78400000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.94399999999999995,"15min":0.94099999999999995},"min":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.222,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2549999999999999,"5min":2.452,"15min":2.452},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.008,"15min":1.024},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2050000000000001,"5min":2.649,"15min":2.649},"last":1.087},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2050000000000001,"5min":2.077,"15min":2.077},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.95799999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2350000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.1220000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2509999999999999,"5min":2.395,"15min":2.395},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.149,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.93700000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.286,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.198,"15min":1.21},"min":{"1min":0.82099999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6100000000000001,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.292,"15min":1.3160000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.3080000000000001,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.93300000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.671},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3720000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.034},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.2769999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.5740000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.778},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.645}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.706},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":2.1920000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.752},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.47}]}]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.81699999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.625},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.88900000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.0049999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.171},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.052}]},{"osd":7,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.323}]}]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57299999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.266,"15min":1.266},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.501,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.546,"15min":1.546},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90800000000000003,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.288,"5min":1.49,"15min":1.49},"last":0.44700000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0449999999999999,"15min":1.046},"min":{"1min":0.77700000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.1020000000000001,"15min":1.1120000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.395,"5min":1.871,"15min":1.871},"last":0.63800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.123,"15min":1.141},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.282,"5min":2.302,"15min":2.302},"last":0.876},{"interface":"front","average":{"1min":1.125,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7190000000000001,"5min":1.78,"15min":1.78},"last":0.92200000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2370000000000001,"15min":1.2390000000000001},"min":{"1min":0.94499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4790000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.048},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.452,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.2909999999999999},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4390000000000001,"5min":2.282,"15min":2.282},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.339,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.98799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302448,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.139,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.873}]},{"osd":2,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.59499999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0549999999999999,"5min":1.333,"15min":1.333},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1779999999999999,"5min":1.252,"15min":1.252},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.546,"15min":1.546},"last":0.90500000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4119999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.006},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.087,"15min":1.093},"min":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.93300000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1739999999999999,"15min":1.1759999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.04},{"interface":"front","average":{"1min":1.123,"5min":1.206,"15min":1.22},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4339999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.573,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2669999999999999,"15min":1.2729999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5249999999999999,"5min":2.093,"15min":2.093},"last":1.256}]},{"osd":7,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.4930000000000001,"5min":2.524,"15min":2.524},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.91800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6519999999999999,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.228}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.59599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76800000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92100000000000004,"15min":0.91200000000000003},"min":{"1min":0.77100000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.746},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.96599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.88700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.343,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2}]},{"osd":5,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1000000000000001,"15min":1.101},"min":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.421,"5min":1.51,"15min":1.51},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.175,"15min":1.169},"min":{"1min":0.91700000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7669999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2070000000000001,"15min":1.2010000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.325},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.577,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.081}]},{"osd":7,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.286,"15min":1.276},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.238}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:23:22.556 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:23:22.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 Processor -- start 2023-10-07T23:23:22.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 -- start start 2023-10-07T23:23:22.669 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90cc161a70 0x7f90cc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f90cc15d2a0 0x7f90cc0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc06bbf0 con 0x7f90cc15ced0 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0c9010 con 0x7f90cc161a70 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cb59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc06b020 con 0x7f90cc15d2a0 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90c9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90cc161a70 0x7f90cc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90c9d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f90cc161a70 0x7f90cc06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44582/0 (socket says 172.21.15.57:44582) 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90c9d9b700 1 -- 172.21.15.57:0/951033555 learned_addr learned my addr 172.21.15.57:0/951033555 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.664+0000 7f90cad9d700 1 --2- 172.21.15.57:0/951033555 >> v2:172.21.15.154:3300/0 conn(0x7f90cc15d2a0 0x7f90cc0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c9d9b700 1 -- 172.21.15.57:0/951033555 >> v2:172.21.15.154:3300/0 conn(0x7f90cc15d2a0 msgr2=0x7f90cc0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c9d9b700 1 --2- 172.21.15.57:0/951033555 >> v2:172.21.15.154:3300/0 conn(0x7f90cc15d2a0 0x7f90cc0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.670 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c9d9b700 1 -- 172.21.15.57:0/951033555 >> v1:172.21.15.154:6789/0 conn(0x7f90cc15ced0 legacy=0x7f90cc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:22.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c9d9b700 1 -- 172.21.15.57:0/951033555 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc06b2c0 con 0x7f90cc161a70 2023-10-07T23:23:22.671 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c9d9b700 1 --2- 172.21.15.57:0/951033555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc161a70 0x7f90cc06c080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0cb0e0 tx=0x7f90b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584fde9b66014ca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c959a700 1 -- 172.21.15.57:0/951033555 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b400e070 con 0x7f90cc161a70 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c959a700 1 -- 172.21.15.57:0/951033555 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b400ba10 con 0x7f90cc161a70 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90c959a700 1 -- 172.21.15.57:0/951033555 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40092e0 con 0x7f90cc161a70 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- 172.21.15.57:0/951033555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc161a70 msgr2=0x7f90cc06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- 172.21.15.57:0/951033555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc161a70 0x7f90cc06c080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0cb0e0 tx=0x7f90b4009840 comp rx=0 tx=0).stop 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- 172.21.15.57:0/951033555 shutdown_connections 2023-10-07T23:23:22.672 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- 172.21.15.57:0/951033555 >> v2:172.21.15.154:3300/0 conn(0x7f90cc15d2a0 0x7f90cc0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- 172.21.15.57:0/951033555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc161a70 0x7f90cc06c080 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- 172.21.15.57:0/951033555 >> 172.21.15.57:0/951033555 conn(0x7f90cc0670e0 msgr2=0x7f90cc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- 172.21.15.57:0/951033555 shutdown_connections 2023-10-07T23:23:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- 172.21.15.57:0/951033555 wait complete. 2023-10-07T23:23:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 Processor -- start 2023-10-07T23:23:22.674 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 -- start start 2023-10-07T23:23:22.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc15ced0 0x7f90cc15f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.675 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.668+0000 7f90cb59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc161a70 0x7f90cc14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc154ea0 con 0x7f90cc15d2a0 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc155030 con 0x7f90cc161a70 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc15e8e0 con 0x7f90cc15ced0 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44586/0 (socket says 172.21.15.57:44586) 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 -- 172.21.15.57:0/3076064841 learned_addr learned my addr 172.21.15.57:0/3076064841 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cad9d700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc161a70 0x7f90cc14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90ca59c700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc15ced0 0x7f90cc15f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 -- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc15ced0 msgr2=0x7f90cc15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc15ced0 0x7f90cc15f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 -- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc161a70 msgr2=0x7f90cc14fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.676 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc161a70 0x7f90cc14fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 -- 172.21.15.57:0/3076064841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4009ae0 con 0x7f90cc15d2a0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c9d9b700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90cc06ec20 tx=0x7f90b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4012920 con 0x7f90cc15d2a0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4012a80 con 0x7f90cc15d2a0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4012e20 con 0x7f90cc15d2a0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc15ebc0 con 0x7f90cc15d2a0 2023-10-07T23:23:22.677 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc15f180 con 0x7f90cc15d2a0 2023-10-07T23:23:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc1551e0 con 0x7f90cc15d2a0 2023-10-07T23:23:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.672+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f90b401d630 con 0x7f90cc15d2a0 2023-10-07T23:23:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.676+0000 7f90c27fc700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 0x7f90ac062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.676+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90b407dd10 con 0x7f90cc15d2a0 2023-10-07T23:23:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.676+0000 7f90ca59c700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 0x7f90ac062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:22.680 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.676+0000 7f90ca59c700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 0x7f90ac062a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0c0b00 tx=0x7f90bc009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:22.688 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.684+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f90cc1551e0 con 0x7f90cc15d2a0 2023-10-07T23:23:22.850 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.848+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90cc1551a0 con 0x7f90ac060550 2023-10-07T23:23:22.866 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.864+0000 7f90c27fc700 1 -- 172.21.15.57:0/3076064841 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272613 (secure 0 0 0) 0x7f90cc1551a0 con 0x7f90ac060550 2023-10-07T23:23:22.871 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 msgr2=0x7f90ac062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 0x7f90ac062a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0c0b00 tx=0x7f90bc009060 comp rx=0 tx=0).stop 2023-10-07T23:23:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 msgr2=0x7f90cc15e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f90cc06ec20 tx=0x7f90b4017000 comp rx=0 tx=0).stop 2023-10-07T23:23:22.874 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 shutdown_connections 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f90ac060550 0x7f90ac062a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90cc161a70 0x7f90cc14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90cc15d2a0 0x7f90cc15e1e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 --2- 172.21.15.57:0/3076064841 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90cc15ced0 0x7f90cc15f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 >> 172.21.15.57:0/3076064841 conn(0x7f90cc0670e0 msgr2=0x7f90cc150d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 shutdown_connections 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:22.868+0000 7f90cb59e700 1 -- 172.21.15.57:0/3076064841 wait complete. 2023-10-07T23:23:22.875 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:23:22.887 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-10-07T23:23:21.110681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2399720,"kb_used_data":5032,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752575000,"statfs":{"total":773094113280,"available":770636800000,"internally_reserved":0,"allocated":5152768,"data_stored":2718704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001284"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:18.119428+0000","last_change":"2023-10-07T23:23:18.119428+0000","last_active":"2023-10-07T23:23:18.119428+0000","last_peered":"2023-10-07T23:23:18.119428+0000","last_clean":"2023-10-07T23:23:18.119428+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:18.119428+0000","last_undegraded":"2023-10-07T23:23:18.119428+0000","last_fullsized":"2023-10-07T23:23:18.119428+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-07T23:23:17.102380+0000","last_change":"2023-10-07T23:23:17.102380+0000","last_active":"2023-10-07T23:23:17.102380+0000","last_peered":"2023-10-07T23:23:17.102380+0000","last_clean":"2023-10-07T23:23:17.102380+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-07T23:23:17.102380+0000","last_undegraded":"2023-10-07T23:23:17.102380+0000","last_fullsized":"2023-10-07T23:23:17.102380+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-10-07T23:23:14.097236+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:23:14.097236+0000","last_peered":"2023-10-07T23:23:14.097236+0000","last_clean":"2023-10-07T23:23:14.097236+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:23:14.097236+0000","last_undegraded":"2023-10-07T23:23:14.097236+0000","last_fullsized":"2023-10-07T23:23:14.097236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298352,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073488,"statfs":{"total":96636764160,"available":96331251712,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95199999999999996,"15min":0.94999999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.216,"5min":1.258,"15min":1.258},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3149999999999999,"5min":1.508,"15min":1.508},"last":0.53900000000000003}]},{"osd":1,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.872,"15min":0.872},"min":{"1min":0.67100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.04,"5min":1.302,"15min":1.302},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94399999999999995,"15min":0.95299999999999996},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3660000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.046},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.327,"5min":1.365,"15min":1.365},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.056,"5min":1.0720000000000001,"15min":1.075},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.341,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.92000000000000004}]},{"osd":3,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1619999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2549999999999999,"5min":1.51,"15min":1.51},"last":0.872}]},{"osd":4,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.385,"5min":1.603,"15min":1.603},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.107,"15min":1.127},"min":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2809999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.194,"15min":1.216},"min":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.905,"15min":1.905},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.2270000000000001},"min":{"1min":0.85299999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.341,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.232,"15min":1.24},"min":{"1min":0.91300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6160000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.96199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":1.054},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.012}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.115},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82899999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94399999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.89600000000000002}]},{"osd":4,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.196}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.157}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.236}]}]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0489999999999999,"15min":1.052},"min":{"1min":0.89100000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2909999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0249999999999999}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":0.97699999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0600000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88,"15min":0.878},"min":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1299999999999999,"5min":1.296,"15min":1.296},"last":0.78400000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.94399999999999995,"15min":0.94099999999999995},"min":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.222,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2549999999999999,"5min":2.452,"15min":2.452},"last":0.63700000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.008,"15min":1.024},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2050000000000001,"5min":2.649,"15min":2.649},"last":1.087},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2050000000000001,"5min":2.077,"15min":2.077},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.95799999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2350000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.1220000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2509999999999999,"5min":2.395,"15min":2.395},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.149,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.93700000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.286,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.99099999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.198,"15min":1.21},"min":{"1min":0.82099999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6100000000000001,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":0.879},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.292,"15min":1.3160000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.3080000000000001,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":0.93300000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297772,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074068,"statfs":{"total":96636764160,"available":96331845632,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.671},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.3720000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.034},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.2769999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.5740000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.778},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.645}]},{"osd":5,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.706},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":2.1920000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:22:16 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.752},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.47}]}]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.81699999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.625},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.88900000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.0049999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.171},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.052}]},{"osd":7,"last update":"Sat Oct 7 23:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.196,"15min":1.196},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.323}]}]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.57299999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.266,"15min":1.266},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.501,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.546,"15min":1.546},"last":0.32500000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90800000000000003,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.288,"5min":1.49,"15min":1.49},"last":0.44700000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0449999999999999,"15min":1.046},"min":{"1min":0.77700000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.1020000000000001,"15min":1.1120000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.395,"5min":1.871,"15min":1.871},"last":0.63800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.123,"15min":1.141},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.282,"5min":2.302,"15min":2.302},"last":0.876},{"interface":"front","average":{"1min":1.125,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7190000000000001,"5min":1.78,"15min":1.78},"last":0.92200000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2370000000000001,"15min":1.2390000000000001},"min":{"1min":0.94499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4790000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.048},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.452,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.2909999999999999},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4390000000000001,"5min":2.282,"15min":2.282},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.339,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.98799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302448,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069392,"statfs":{"total":96636764160,"available":96327057408,"internally_reserved":0,"allocated":1089536,"data_stored":782614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.139,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.873}]},{"osd":2,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.59499999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0549999999999999,"5min":1.333,"15min":1.333},"last":0.59499999999999997}]},{"osd":3,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1779999999999999,"5min":1.252,"15min":1.252},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.546,"15min":1.546},"last":0.90500000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4119999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.006},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.087,"15min":1.093},"min":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.93300000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1739999999999999,"15min":1.1759999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.04},{"interface":"front","average":{"1min":1.123,"5min":1.206,"15min":1.22},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4339999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":1.198}]},{"osd":6,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.573,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2669999999999999,"15min":1.2729999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5249999999999999,"5min":2.093,"15min":2.093},"last":1.256}]},{"osd":7,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.4930000000000001,"5min":2.524,"15min":2.524},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.91800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6519999999999999,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.228}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301868,"kb_used_data":484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069972,"statfs":{"total":96636764160,"available":96327651328,"internally_reserved":0,"allocated":495616,"data_stored":192246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.59599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.76800000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92100000000000004,"15min":0.91200000000000003},"min":{"1min":0.77100000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.746},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.96599999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.88700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.343,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2}]},{"osd":5,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1000000000000001,"15min":1.101},"min":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.421,"5min":1.51,"15min":1.51},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.175,"15min":1.169},"min":{"1min":0.91700000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7669999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2070000000000001,"15min":1.2010000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.325},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.577,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.081}]},{"osd":7,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.286,"15min":1.276},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.238}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:23:22.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-10-07T23:23:22.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-10-07T23:23:22.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-10-07T23:23:22.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-10-07T23:23:22.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-10-07T23:23:22.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-10-07T23:23:22.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-10-07T23:23:22.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-10-07T23:23:22.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-10-07T23:23:22.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-10-07T23:23:22.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-10-07T23:23:22.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-10-07T23:23:22.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-10-07T23:23:22.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-10-07T23:23:22.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-10-07T23:23:22.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:18.119428+0000', 'last_change': '2023-10-07T23:23:18.119428+0000', 'last_active': '2023-10-07T23:23:18.119428+0000', 'last_peered': '2023-10-07T23:23:18.119428+0000', 'last_clean': '2023-10-07T23:23:18.119428+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:18.119428+0000', 'last_undegraded': '2023-10-07T23:23:18.119428+0000', 'last_fullsized': '2023-10-07T23:23:18.119428+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:18.119428+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-10-07T23:23:22.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-07T23:23:17.102380+0000', 'last_change': '2023-10-07T23:23:17.102380+0000', 'last_active': '2023-10-07T23:23:17.102380+0000', 'last_peered': '2023-10-07T23:23:17.102380+0000', 'last_clean': '2023-10-07T23:23:17.102380+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-07T23:23:17.102380+0000', 'last_undegraded': '2023-10-07T23:23:17.102380+0000', 'last_fullsized': '2023-10-07T23:23:17.102380+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'last_clean_scrub_stamp': '2023-10-07T23:23:17.102380+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-07T23:23:22.921 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-07T23:23:23.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.020+0000 7f8fb285c700 1 Processor -- start 2023-10-07T23:23:23.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.020+0000 7f8fb285c700 1 -- start start 2023-10-07T23:23:23.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.020+0000 7f8fb285c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac158c90 0x7f8fac159060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb285c700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac15dac0 con 0x7f8fac0cae70 2023-10-07T23:23:23.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb285c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac15cf40 con 0x7f8fac159fd0 2023-10-07T23:23:23.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb285c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac15d220 con 0x7f8fac158c90 2023-10-07T23:23:23.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb0df9700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8fac0cae70 0x7f8fac068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:45838/0 (socket says 172.21.15.57:45838) 2023-10-07T23:23:23.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1983201835 learned_addr learned my addr 172.21.15.57:0/1983201835 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:23.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fabfff700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac158c90 0x7f8fac159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.028 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226793652 0 0) 0x7f8fac15dac0 con 0x7f8fac0cae70 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f98003570 con 0x7f8fac0cae70 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768098085 0 0) 0x7f8fac15cf40 con 0x7f8fac159fd0 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fac15dac0 con 0x7f8fac159fd0 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445849328 0 0) 0x7f8f98003570 con 0x7f8fac0cae70 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac158c90 msgr2=0x7f8fac159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac158c90 0x7f8fac159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 >> v1:172.21.15.154:6789/0 conn(0x7f8fac159fd0 legacy=0x7f8fac0ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac15d500 con 0x7f8fac0cae70 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f8fa40017f0 con 0x7f8fac0cae70 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 >> v1:172.21.15.57:6789/0 conn(0x7f8fac0cae70 legacy=0x7f8fac068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f98003c20 0x7f8f98004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f98004810 0x7f8f98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f980051c0 0x7f8f9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac15d220 con 0x7f8f98003c20 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9800c0b0 con 0x7f8f980051c0 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9800c270 con 0x7f8f98004810 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fabfff700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac158c90 0x7f8fac159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f98004810 0x7f8f98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fb0df9700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f980051c0 0x7f8f9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fabfff700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f98003c20 0x7f8f98004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 -- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f980051c0 msgr2=0x7f8f9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:23.030 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f980051c0 0x7f8f9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 -- 172.21.15.57:0/1983201835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f98003c20 msgr2=0x7f8f98004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f98003c20 0x7f8f98004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 -- 172.21.15.57:0/1983201835 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f98003570 con 0x7f8f98004810 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8fab7fe700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f98004810 0x7f8f98004c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001a10 tx=0x7f8f94007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed354644d46964b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400c5d0 con 0x7f8f98004810 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400c730 con 0x7f8f98004810 2023-10-07T23:23:23.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.024+0000 7f8faaffd700 1 -- 172.21.15.57:0/1983201835 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400caf0 con 0x7f8f98004810 2023-10-07T23:23:23.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f98004810 msgr2=0x7f8f98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1983201835 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8f98004810 0x7f8f98004c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001a10 tx=0x7f8f94007d10 comp rx=0 tx=0).stop 2023-10-07T23:23:23.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1983201835 reap_dead start 2023-10-07T23:23:23.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- 172.21.15.57:0/1983201835 shutdown_connections 2023-10-07T23:23:23.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- 172.21.15.57:0/1983201835 >> 172.21.15.57:0/1983201835 conn(0x7f8fac0cce30 msgr2=0x7f8fac0c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:23.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- 172.21.15.57:0/1983201835 shutdown_connections 2023-10-07T23:23:23.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- 172.21.15.57:0/1983201835 wait complete. 2023-10-07T23:23:23.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 Processor -- start 2023-10-07T23:23:23.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.028+0000 7f8fb285c700 1 -- start start 2023-10-07T23:23:23.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 0x7f8fac0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac0b0830 0x7f8fac156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0c78f0 con 0x7f8fac0c6f40 2023-10-07T23:23:23.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0b1e20 con 0x7f8f98004810 2023-10-07T23:23:23.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0b0d80 con 0x7f8fac0b0830 2023-10-07T23:23:23.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 0x7f8fac0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44598/0 (socket says 172.21.15.57:44598) 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1578249113 learned_addr learned my addr 172.21.15.57:0/1578249113 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac0b0830 0x7f8fac156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 0x7f8fac0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:33714/0 (socket says 172.21.15.57:33714) 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac0b0830 msgr2=0x7f8fac156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac0b0830 0x7f8fac156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 msgr2=0x7f8fac0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 0x7f8fac0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 -- 172.21.15.57:0/1578249113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0b1060 con 0x7f8fac0c6f40 2023-10-07T23:23:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb0df9700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0cb560 tx=0x7f8fa4002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa400a9b0 con 0x7f8fac0c6f40 2023-10-07T23:23:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa400ab30 con 0x7f8fac0c6f40 2023-10-07T23:23:23.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac0b1340 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40133f0 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.032+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac0b1900 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac0b1b20 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fa4013820 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fa3fff700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 0x7f8f8c062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fa407d070 con 0x7f8fac0c6f40 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fabfff700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 0x7f8f8c062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.036+0000 7f8fabfff700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 0x7f8f8c062a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0c3180 tx=0x7f8f9c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.050 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.048+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fa404d5c0 con 0x7f8fac0c6f40 2023-10-07T23:23:23.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.268+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8fac0711c0 con 0x7f8fac0c6f40 2023-10-07T23:23:23.273 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.268+0000 7f8fa3fff700 1 -- 172.21.15.57:0/1578249113 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1015 (secure 0 0 0) 0x7f8fa4017080 con 0x7f8fac0c6f40 2023-10-07T23:23:23.273 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:23.273 INFO:teuthology.orchestra.run.smithi057.stdout:{"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1696720871,"num_in_osds":8,"osd_in_since":1696720862,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2457747456,"bytes_avail":770636365824,"bytes_total":773094113280,"write_bytes_sec":2222,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":19,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5359}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-07T23:23:07.106662+0000","services":{}},"progress_events":{}} 2023-10-07T23:23:23.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 msgr2=0x7f8f8c062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 0x7f8f8c062a30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0c3180 tx=0x7f8f9c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 msgr2=0x7f8fac0c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0cb560 tx=0x7f8fa4002170 comp rx=0 tx=0).stop 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 shutdown_connections 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fac0c6f40 0x7f8fac0c73b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fac0b0830 0x7f8fac156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8f8c060580 0x7f8f8c062a30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 --2- 172.21.15.57:0/1578249113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f98004810 0x7f8fac0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 >> 172.21.15.57:0/1578249113 conn(0x7f8fac0cce30 msgr2=0x7f8fac157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 shutdown_connections 2023-10-07T23:23:23.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.272+0000 7f8fb285c700 1 -- 172.21.15.57:0/1578249113 wait complete. 2023-10-07T23:23:23.287 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-07T23:23:23.288 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:23:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 Processor -- start 2023-10-07T23:23:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 -- start start 2023-10-07T23:23:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88e015a210 0x7f88e00c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e00ca6c0 con 0x7f88e00cb6d0 2023-10-07T23:23:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e00cb240 con 0x7f88e015a210 2023-10-07T23:23:23.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00ca910 con 0x7f88e0155780 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:55464/0 (socket says 172.21.15.57:55464) 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e4dbb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88e015a210 0x7f88e00c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 -- 172.21.15.57:0/156498443 learned_addr learned my addr 172.21.15.57:0/156498443 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 -- 172.21.15.57:0/156498443 >> v2:172.21.15.154:3300/0 conn(0x7f88e015a210 msgr2=0x7f88e00c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 --2- 172.21.15.57:0/156498443 >> v2:172.21.15.154:3300/0 conn(0x7f88e015a210 0x7f88e00c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 -- 172.21.15.57:0/156498443 >> v1:172.21.15.154:6789/0 conn(0x7f88e00cb6d0 legacy=0x7f88e0069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 -- 172.21.15.57:0/156498443 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00cabf0 con 0x7f88e0155780 2023-10-07T23:23:23.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88dffff700 1 --2- 172.21.15.57:0/156498443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009990 tx=0x7f88d0009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=532541f2198541d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88deffd700 1 -- 172.21.15.57:0/156498443 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0009e80 con 0x7f88e0155780 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88deffd700 1 -- 172.21.15.57:0/156498443 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d0005c10 con 0x7f88e0155780 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88deffd700 1 -- 172.21.15.57:0/156498443 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d000b650 con 0x7f88e0155780 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 -- 172.21.15.57:0/156498443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 msgr2=0x7f88e0155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.388+0000 7f88e55bc700 1 --2- 172.21.15.57:0/156498443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009990 tx=0x7f88d0009220 comp rx=0 tx=0).stop 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- 172.21.15.57:0/156498443 shutdown_connections 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 --2- 172.21.15.57:0/156498443 >> v2:172.21.15.154:3300/0 conn(0x7f88e015a210 0x7f88e00c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 --2- 172.21.15.57:0/156498443 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e0155b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- 172.21.15.57:0/156498443 >> 172.21.15.57:0/156498443 conn(0x7f88e006b5d0 msgr2=0x7f88e006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- 172.21.15.57:0/156498443 shutdown_connections 2023-10-07T23:23:23.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- 172.21.15.57:0/156498443 wait complete. 2023-10-07T23:23:23.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 Processor -- start 2023-10-07T23:23:23.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- start start 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e00cb6d0 0x7f88e00b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e00c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00b27e0 con 0x7f88e015a210 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00b0e40 con 0x7f88e00cb6d0 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e55bc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e015ec80 con 0x7f88e0155780 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e4dbb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e4dbb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44610/0 (socket says 172.21.15.57:44610) 2023-10-07T23:23:23.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88e4dbb700 1 -- 172.21.15.57:0/1863074204 learned_addr learned my addr 172.21.15.57:0/1863074204 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:23.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88dffff700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e00cb6d0 0x7f88e00b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.398 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.392+0000 7f88df7fe700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e00c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 -- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 msgr2=0x7f88e00c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e00c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 -- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e00cb6d0 msgr2=0x7f88e00b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e00cb6d0 0x7f88e00b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 -- 172.21.15.57:0/1863074204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0009480 con 0x7f88e015a210 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e4dbb700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009e90 tx=0x7f88d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800cb70 con 0x7f88e015a210 2023-10-07T23:23:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800f040 con 0x7f88e015a210 2023-10-07T23:23:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e015ef60 con 0x7f88e015a210 2023-10-07T23:23:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8014480 con 0x7f88e015a210 2023-10-07T23:23:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e015f520 con 0x7f88e015a210 2023-10-07T23:23:23.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.396+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e00c7210 con 0x7f88e015a210 2023-10-07T23:23:23.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.400+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f88d8014830 con 0x7f88e015a210 2023-10-07T23:23:23.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.400+0000 7f88c7fff700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 0x7f88cc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:23.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.400+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f88d807d430 con 0x7f88e015a210 2023-10-07T23:23:23.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.400+0000 7f88dffff700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 0x7f88cc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:23.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.400+0000 7f88dffff700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 0x7f88cc062a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009260 tx=0x7f88d0016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:23.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.408+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f88e00c7210 con 0x7f88e015a210 2023-10-07T23:23:23.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.564+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88e00c7210 con 0x7f88cc060550 2023-10-07T23:23:23.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.580+0000 7f88c7fff700 1 -- 172.21.15.57:0/1863074204 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287171 (secure 0 0 0) 0x7f88e00c7210 con 0x7f88cc060550 2023-10-07T23:23:23.587 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 msgr2=0x7f88cc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 0x7f88cc062a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009260 tx=0x7f88d0016000 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 msgr2=0x7f88e015fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009e90 tx=0x7f88d800b040 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 shutdown_connections 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f88e015a210 0x7f88e015fa20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f88e0155780 0x7f88e00c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f88cc060550 0x7f88cc062a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 --2- 172.21.15.57:0/1863074204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88e00cb6d0 0x7f88e00b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:23.590 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 >> 172.21.15.57:0/1863074204 conn(0x7f88e006b5d0 msgr2=0x7f88e006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:23.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 shutdown_connections 2023-10-07T23:23:23.591 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:23.584+0000 7f88e55bc700 1 -- 172.21.15.57:0/1863074204 wait complete. 2023-10-07T23:23:23.591 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:23:23.603 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-10-07T23:23:23.111248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400144,"kb_used_data":5456,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752574576,"statfs":{"total":773094113280,"available":770636365824,"internally_reserved":0,"allocated":5586944,"data_stored":2911628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988830"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184880+0000","last_change":"2023-10-07T23:23:19.178860+0000","last_active":"2023-10-07T23:23:19.184880+0000","last_peered":"2023-10-07T23:23:19.184880+0000","last_clean":"2023-10-07T23:23:19.184880+0000","last_became_active":"2023-10-07T23:23:19.178526+0000","last_became_peered":"2023-10-07T23:23:19.178526+0000","last_unstale":"2023-10-07T23:23:19.184880+0000","last_undegraded":"2023-10-07T23:23:19.184880+0000","last_fullsized":"2023-10-07T23:23:19.184880+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189325+0000","last_change":"2023-10-07T23:23:19.183620+0000","last_active":"2023-10-07T23:23:19.189325+0000","last_peered":"2023-10-07T23:23:19.189325+0000","last_clean":"2023-10-07T23:23:19.189325+0000","last_became_active":"2023-10-07T23:23:19.183131+0000","last_became_peered":"2023-10-07T23:23:19.183131+0000","last_unstale":"2023-10-07T23:23:19.189325+0000","last_undegraded":"2023-10-07T23:23:19.189325+0000","last_fullsized":"2023-10-07T23:23:19.189325+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189364+0000","last_change":"2023-10-07T23:23:19.178596+0000","last_active":"2023-10-07T23:23:19.189364+0000","last_peered":"2023-10-07T23:23:19.189364+0000","last_clean":"2023-10-07T23:23:19.189364+0000","last_became_active":"2023-10-07T23:23:19.178289+0000","last_became_peered":"2023-10-07T23:23:19.178289+0000","last_unstale":"2023-10-07T23:23:19.189364+0000","last_undegraded":"2023-10-07T23:23:19.189364+0000","last_fullsized":"2023-10-07T23:23:19.189364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198239+0000","last_change":"2023-10-07T23:23:19.185639+0000","last_active":"2023-10-07T23:23:19.198239+0000","last_peered":"2023-10-07T23:23:19.198239+0000","last_clean":"2023-10-07T23:23:19.198239+0000","last_became_active":"2023-10-07T23:23:19.185238+0000","last_became_peered":"2023-10-07T23:23:19.185238+0000","last_unstale":"2023-10-07T23:23:19.198239+0000","last_undegraded":"2023-10-07T23:23:19.198239+0000","last_fullsized":"2023-10-07T23:23:19.198239+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:04.524579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189159+0000","last_change":"2023-10-07T23:23:19.182456+0000","last_active":"2023-10-07T23:23:19.189159+0000","last_peered":"2023-10-07T23:23:19.189159+0000","last_clean":"2023-10-07T23:23:19.189159+0000","last_became_active":"2023-10-07T23:23:19.182160+0000","last_became_peered":"2023-10-07T23:23:19.182160+0000","last_unstale":"2023-10-07T23:23:19.189159+0000","last_undegraded":"2023-10-07T23:23:19.189159+0000","last_fullsized":"2023-10-07T23:23:19.189159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183591+0000","last_change":"2023-10-07T23:23:19.174227+0000","last_active":"2023-10-07T23:23:19.183591+0000","last_peered":"2023-10-07T23:23:19.183591+0000","last_clean":"2023-10-07T23:23:19.183591+0000","last_became_active":"2023-10-07T23:23:19.173826+0000","last_became_peered":"2023-10-07T23:23:19.173826+0000","last_unstale":"2023-10-07T23:23:19.183591+0000","last_undegraded":"2023-10-07T23:23:19.183591+0000","last_fullsized":"2023-10-07T23:23:19.183591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:57.476739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.194529+0000","last_change":"2023-10-07T23:23:19.190038+0000","last_active":"2023-10-07T23:23:19.194529+0000","last_peered":"2023-10-07T23:23:19.194529+0000","last_clean":"2023-10-07T23:23:19.194529+0000","last_became_active":"2023-10-07T23:23:19.189767+0000","last_became_peered":"2023-10-07T23:23:19.189767+0000","last_unstale":"2023-10-07T23:23:19.194529+0000","last_undegraded":"2023-10-07T23:23:19.194529+0000","last_fullsized":"2023-10-07T23:23:19.194529+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198287+0000","last_change":"2023-10-07T23:23:19.186078+0000","last_active":"2023-10-07T23:23:19.198287+0000","last_peered":"2023-10-07T23:23:19.198287+0000","last_clean":"2023-10-07T23:23:19.198287+0000","last_became_active":"2023-10-07T23:23:19.185764+0000","last_became_peered":"2023-10-07T23:23:19.185764+0000","last_unstale":"2023-10-07T23:23:19.198287+0000","last_undegraded":"2023-10-07T23:23:19.198287+0000","last_fullsized":"2023-10-07T23:23:19.198287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198528+0000","last_change":"2023-10-07T23:23:19.189011+0000","last_active":"2023-10-07T23:23:19.198528+0000","last_peered":"2023-10-07T23:23:19.198528+0000","last_clean":"2023-10-07T23:23:19.198528+0000","last_became_active":"2023-10-07T23:23:19.188690+0000","last_became_peered":"2023-10-07T23:23:19.188690+0000","last_unstale":"2023-10-07T23:23:19.198528+0000","last_undegraded":"2023-10-07T23:23:19.198528+0000","last_fullsized":"2023-10-07T23:23:19.198528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198473+0000","last_change":"2023-10-07T23:23:19.186540+0000","last_active":"2023-10-07T23:23:19.198473+0000","last_peered":"2023-10-07T23:23:19.198473+0000","last_clean":"2023-10-07T23:23:19.198473+0000","last_became_active":"2023-10-07T23:23:19.184407+0000","last_became_peered":"2023-10-07T23:23:19.184407+0000","last_unstale":"2023-10-07T23:23:19.198473+0000","last_undegraded":"2023-10-07T23:23:19.198473+0000","last_fullsized":"2023-10-07T23:23:19.198473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.192965+0000","last_change":"2023-10-07T23:23:19.182230+0000","last_active":"2023-10-07T23:23:19.192965+0000","last_peered":"2023-10-07T23:23:19.192965+0000","last_clean":"2023-10-07T23:23:19.192965+0000","last_became_active":"2023-10-07T23:23:19.181888+0000","last_became_peered":"2023-10-07T23:23:19.181888+0000","last_unstale":"2023-10-07T23:23:19.192965+0000","last_undegraded":"2023-10-07T23:23:19.192965+0000","last_fullsized":"2023-10-07T23:23:19.192965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:45.694357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189044+0000","last_change":"2023-10-07T23:23:19.181495+0000","last_active":"2023-10-07T23:23:19.189044+0000","last_peered":"2023-10-07T23:23:19.189044+0000","last_clean":"2023-10-07T23:23:19.189044+0000","last_became_active":"2023-10-07T23:23:19.180814+0000","last_became_peered":"2023-10-07T23:23:19.180814+0000","last_unstale":"2023-10-07T23:23:19.189044+0000","last_undegraded":"2023-10-07T23:23:19.189044+0000","last_fullsized":"2023-10-07T23:23:19.189044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188921+0000","last_change":"2023-10-07T23:23:19.184004+0000","last_active":"2023-10-07T23:23:19.188921+0000","last_peered":"2023-10-07T23:23:19.188921+0000","last_clean":"2023-10-07T23:23:19.188921+0000","last_became_active":"2023-10-07T23:23:19.183539+0000","last_became_peered":"2023-10-07T23:23:19.183539+0000","last_unstale":"2023-10-07T23:23:19.188921+0000","last_undegraded":"2023-10-07T23:23:19.188921+0000","last_fullsized":"2023-10-07T23:23:19.188921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:33.705262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193170+0000","last_change":"2023-10-07T23:23:19.182254+0000","last_active":"2023-10-07T23:23:19.193170+0000","last_peered":"2023-10-07T23:23:19.193170+0000","last_clean":"2023-10-07T23:23:19.193170+0000","last_became_active":"2023-10-07T23:23:19.181917+0000","last_became_peered":"2023-10-07T23:23:19.181917+0000","last_unstale":"2023-10-07T23:23:19.193170+0000","last_undegraded":"2023-10-07T23:23:19.193170+0000","last_fullsized":"2023-10-07T23:23:19.193170+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:27.631972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186689+0000","last_change":"2023-10-07T23:23:19.177405+0000","last_active":"2023-10-07T23:23:19.186689+0000","last_peered":"2023-10-07T23:23:19.186689+0000","last_clean":"2023-10-07T23:23:19.186689+0000","last_became_active":"2023-10-07T23:23:19.177107+0000","last_became_peered":"2023-10-07T23:23:19.177107+0000","last_unstale":"2023-10-07T23:23:19.186689+0000","last_undegraded":"2023-10-07T23:23:19.186689+0000","last_fullsized":"2023-10-07T23:23:19.186689+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:45.765891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186358+0000","last_change":"2023-10-07T23:23:19.181580+0000","last_active":"2023-10-07T23:23:19.186358+0000","last_peered":"2023-10-07T23:23:19.186358+0000","last_clean":"2023-10-07T23:23:19.186358+0000","last_became_active":"2023-10-07T23:23:19.181173+0000","last_became_peered":"2023-10-07T23:23:19.181173+0000","last_unstale":"2023-10-07T23:23:19.186358+0000","last_undegraded":"2023-10-07T23:23:19.186358+0000","last_fullsized":"2023-10-07T23:23:19.186358+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:38.438227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189619+0000","last_change":"2023-10-07T23:23:19.182528+0000","last_active":"2023-10-07T23:23:19.189619+0000","last_peered":"2023-10-07T23:23:19.189619+0000","last_clean":"2023-10-07T23:23:19.189619+0000","last_became_active":"2023-10-07T23:23:19.182260+0000","last_became_peered":"2023-10-07T23:23:19.182260+0000","last_unstale":"2023-10-07T23:23:19.189619+0000","last_undegraded":"2023-10-07T23:23:19.189619+0000","last_fullsized":"2023-10-07T23:23:19.189619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:35.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189752+0000","last_change":"2023-10-07T23:23:19.187175+0000","last_active":"2023-10-07T23:23:19.189752+0000","last_peered":"2023-10-07T23:23:19.189752+0000","last_clean":"2023-10-07T23:23:19.189752+0000","last_became_active":"2023-10-07T23:23:19.186873+0000","last_became_peered":"2023-10-07T23:23:19.186873+0000","last_unstale":"2023-10-07T23:23:19.189752+0000","last_undegraded":"2023-10-07T23:23:19.189752+0000","last_fullsized":"2023-10-07T23:23:19.189752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:27.631972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188810+0000","last_change":"2023-10-07T23:23:19.185113+0000","last_active":"2023-10-07T23:23:19.188810+0000","last_peered":"2023-10-07T23:23:19.188810+0000","last_clean":"2023-10-07T23:23:19.188810+0000","last_became_active":"2023-10-07T23:23:19.184516+0000","last_became_peered":"2023-10-07T23:23:19.184516+0000","last_unstale":"2023-10-07T23:23:19.188810+0000","last_undegraded":"2023-10-07T23:23:19.188810+0000","last_fullsized":"2023-10-07T23:23:19.188810+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:12.196334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188789+0000","last_change":"2023-10-07T23:23:19.181817+0000","last_active":"2023-10-07T23:23:19.188789+0000","last_peered":"2023-10-07T23:23:19.188789+0000","last_clean":"2023-10-07T23:23:19.188789+0000","last_became_active":"2023-10-07T23:23:19.181416+0000","last_became_peered":"2023-10-07T23:23:19.181416+0000","last_unstale":"2023-10-07T23:23:19.188789+0000","last_undegraded":"2023-10-07T23:23:19.188789+0000","last_fullsized":"2023-10-07T23:23:19.188789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:35:11.402150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187491+0000","last_change":"2023-10-07T23:23:19.183970+0000","last_active":"2023-10-07T23:23:19.187491+0000","last_peered":"2023-10-07T23:23:19.187491+0000","last_clean":"2023-10-07T23:23:19.187491+0000","last_became_active":"2023-10-07T23:23:19.183482+0000","last_became_peered":"2023-10-07T23:23:19.183482+0000","last_unstale":"2023-10-07T23:23:19.187491+0000","last_undegraded":"2023-10-07T23:23:19.187491+0000","last_fullsized":"2023-10-07T23:23:19.187491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:41.068514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184182+0000","last_change":"2023-10-07T23:23:19.179860+0000","last_active":"2023-10-07T23:23:19.184182+0000","last_peered":"2023-10-07T23:23:19.184182+0000","last_clean":"2023-10-07T23:23:19.184182+0000","last_became_active":"2023-10-07T23:23:19.178815+0000","last_became_peered":"2023-10-07T23:23:19.178815+0000","last_unstale":"2023-10-07T23:23:19.184182+0000","last_undegraded":"2023-10-07T23:23:19.184182+0000","last_fullsized":"2023-10-07T23:23:19.184182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:01.676402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184918+0000","last_change":"2023-10-07T23:23:19.175457+0000","last_active":"2023-10-07T23:23:19.184918+0000","last_peered":"2023-10-07T23:23:19.184918+0000","last_clean":"2023-10-07T23:23:19.184918+0000","last_became_active":"2023-10-07T23:23:19.175130+0000","last_became_peered":"2023-10-07T23:23:19.175130+0000","last_unstale":"2023-10-07T23:23:19.184918+0000","last_undegraded":"2023-10-07T23:23:19.184918+0000","last_fullsized":"2023-10-07T23:23:19.184918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:05.566152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186204+0000","last_change":"2023-10-07T23:23:19.179985+0000","last_active":"2023-10-07T23:23:19.186204+0000","last_peered":"2023-10-07T23:23:19.186204+0000","last_clean":"2023-10-07T23:23:19.186204+0000","last_became_active":"2023-10-07T23:23:19.179718+0000","last_became_peered":"2023-10-07T23:23:19.179718+0000","last_unstale":"2023-10-07T23:23:19.186204+0000","last_undegraded":"2023-10-07T23:23:19.186204+0000","last_fullsized":"2023-10-07T23:23:19.186204+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186240+0000","last_change":"2023-10-07T23:23:19.179985+0000","last_active":"2023-10-07T23:23:19.186240+0000","last_peered":"2023-10-07T23:23:19.186240+0000","last_clean":"2023-10-07T23:23:19.186240+0000","last_became_active":"2023-10-07T23:23:19.179663+0000","last_became_peered":"2023-10-07T23:23:19.179663+0000","last_unstale":"2023-10-07T23:23:19.186240+0000","last_undegraded":"2023-10-07T23:23:19.186240+0000","last_fullsized":"2023-10-07T23:23:19.186240+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:10.416689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188953+0000","last_change":"2023-10-07T23:23:19.184652+0000","last_active":"2023-10-07T23:23:19.188953+0000","last_peered":"2023-10-07T23:23:19.188953+0000","last_clean":"2023-10-07T23:23:19.188953+0000","last_became_active":"2023-10-07T23:23:19.184340+0000","last_became_peered":"2023-10-07T23:23:19.184340+0000","last_unstale":"2023-10-07T23:23:19.188953+0000","last_undegraded":"2023-10-07T23:23:19.188953+0000","last_fullsized":"2023-10-07T23:23:19.188953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:04.050534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189540+0000","last_change":"2023-10-07T23:23:19.180981+0000","last_active":"2023-10-07T23:23:19.189540+0000","last_peered":"2023-10-07T23:23:19.189540+0000","last_clean":"2023-10-07T23:23:19.189540+0000","last_became_active":"2023-10-07T23:23:19.180650+0000","last_became_peered":"2023-10-07T23:23:19.180650+0000","last_unstale":"2023-10-07T23:23:19.189540+0000","last_undegraded":"2023-10-07T23:23:19.189540+0000","last_fullsized":"2023-10-07T23:23:19.189540+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:10.416689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189073+0000","last_change":"2023-10-07T23:23:19.181724+0000","last_active":"2023-10-07T23:23:19.189073+0000","last_peered":"2023-10-07T23:23:19.189073+0000","last_clean":"2023-10-07T23:23:19.189073+0000","last_became_active":"2023-10-07T23:23:19.181165+0000","last_became_peered":"2023-10-07T23:23:19.181165+0000","last_unstale":"2023-10-07T23:23:19.189073+0000","last_undegraded":"2023-10-07T23:23:19.189073+0000","last_fullsized":"2023-10-07T23:23:19.189073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:58.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186291+0000","last_change":"2023-10-07T23:23:19.179089+0000","last_active":"2023-10-07T23:23:19.186291+0000","last_peered":"2023-10-07T23:23:19.186291+0000","last_clean":"2023-10-07T23:23:19.186291+0000","last_became_active":"2023-10-07T23:23:19.178743+0000","last_became_peered":"2023-10-07T23:23:19.178743+0000","last_unstale":"2023-10-07T23:23:19.186291+0000","last_undegraded":"2023-10-07T23:23:19.186291+0000","last_fullsized":"2023-10-07T23:23:19.186291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:21.095791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187405+0000","last_change":"2023-10-07T23:23:19.175447+0000","last_active":"2023-10-07T23:23:19.187405+0000","last_peered":"2023-10-07T23:23:19.187405+0000","last_clean":"2023-10-07T23:23:19.187405+0000","last_became_active":"2023-10-07T23:23:19.175103+0000","last_became_peered":"2023-10-07T23:23:19.175103+0000","last_unstale":"2023-10-07T23:23:19.187405+0000","last_undegraded":"2023-10-07T23:23:19.187405+0000","last_fullsized":"2023-10-07T23:23:19.187405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:57.895982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183134+0000","last_change":"2023-10-07T23:23:19.175279+0000","last_active":"2023-10-07T23:23:19.183134+0000","last_peered":"2023-10-07T23:23:19.183134+0000","last_clean":"2023-10-07T23:23:19.183134+0000","last_became_active":"2023-10-07T23:23:19.174982+0000","last_became_peered":"2023-10-07T23:23:19.174982+0000","last_unstale":"2023-10-07T23:23:19.183134+0000","last_undegraded":"2023-10-07T23:23:19.183134+0000","last_fullsized":"2023-10-07T23:23:19.183134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:54.500376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.185177+0000","last_change":"2023-10-07T23:23:19.177640+0000","last_active":"2023-10-07T23:23:19.185177+0000","last_peered":"2023-10-07T23:23:19.185177+0000","last_clean":"2023-10-07T23:23:19.185177+0000","last_became_active":"2023-10-07T23:23:19.177273+0000","last_became_peered":"2023-10-07T23:23:19.177273+0000","last_unstale":"2023-10-07T23:23:19.185177+0000","last_undegraded":"2023-10-07T23:23:19.185177+0000","last_fullsized":"2023-10-07T23:23:19.185177+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:01.148050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.194205+0000","last_change":"2023-10-07T23:23:19.184868+0000","last_active":"2023-10-07T23:23:19.194205+0000","last_peered":"2023-10-07T23:23:19.194205+0000","last_clean":"2023-10-07T23:23:19.194205+0000","last_became_active":"2023-10-07T23:23:19.184580+0000","last_became_peered":"2023-10-07T23:23:19.184580+0000","last_unstale":"2023-10-07T23:23:19.194205+0000","last_undegraded":"2023-10-07T23:23:19.194205+0000","last_fullsized":"2023-10-07T23:23:19.194205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:41.992437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.200176+0000","last_change":"2023-10-07T23:23:18.153116+0000","last_active":"2023-10-07T23:23:19.200176+0000","last_peered":"2023-10-07T23:23:19.200176+0000","last_clean":"2023-10-07T23:23:19.200176+0000","last_became_active":"2023-10-07T23:23:18.152781+0000","last_became_peered":"2023-10-07T23:23:18.152781+0000","last_unstale":"2023-10-07T23:23:19.200176+0000","last_undegraded":"2023-10-07T23:23:19.200176+0000","last_fullsized":"2023-10-07T23:23:19.200176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:26.614925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.178325+0000","last_change":"2023-10-07T23:23:19.172880+0000","last_active":"2023-10-07T23:23:19.178325+0000","last_peered":"2023-10-07T23:23:19.178325+0000","last_clean":"2023-10-07T23:23:19.178325+0000","last_became_active":"2023-10-07T23:23:19.172236+0000","last_became_peered":"2023-10-07T23:23:19.172236+0000","last_unstale":"2023-10-07T23:23:19.178325+0000","last_undegraded":"2023-10-07T23:23:19.178325+0000","last_fullsized":"2023-10-07T23:23:19.178325+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:19.000142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199114+0000","last_change":"2023-10-07T23:23:18.153585+0000","last_active":"2023-10-07T23:23:19.199114+0000","last_peered":"2023-10-07T23:23:19.199114+0000","last_clean":"2023-10-07T23:23:19.199114+0000","last_became_active":"2023-10-07T23:23:18.153280+0000","last_became_peered":"2023-10-07T23:23:18.153280+0000","last_unstale":"2023-10-07T23:23:19.199114+0000","last_undegraded":"2023-10-07T23:23:19.199114+0000","last_fullsized":"2023-10-07T23:23:19.199114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:06.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189347+0000","last_change":"2023-10-07T23:23:19.186014+0000","last_active":"2023-10-07T23:23:19.189347+0000","last_peered":"2023-10-07T23:23:19.189347+0000","last_clean":"2023-10-07T23:23:19.189347+0000","last_became_active":"2023-10-07T23:23:19.185745+0000","last_became_peered":"2023-10-07T23:23:19.185745+0000","last_unstale":"2023-10-07T23:23:19.189347+0000","last_undegraded":"2023-10-07T23:23:19.189347+0000","last_fullsized":"2023-10-07T23:23:19.189347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:01.676402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.200479+0000","last_change":"2023-10-07T23:23:18.149433+0000","last_active":"2023-10-07T23:23:19.200479+0000","last_peered":"2023-10-07T23:23:19.200479+0000","last_clean":"2023-10-07T23:23:19.200479+0000","last_became_active":"2023-10-07T23:23:18.149086+0000","last_became_peered":"2023-10-07T23:23:18.149086+0000","last_unstale":"2023-10-07T23:23:19.200479+0000","last_undegraded":"2023-10-07T23:23:19.200479+0000","last_fullsized":"2023-10-07T23:23:19.200479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:29:44.351494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.181344+0000","last_change":"2023-10-07T23:23:19.173457+0000","last_active":"2023-10-07T23:23:19.181344+0000","last_peered":"2023-10-07T23:23:19.181344+0000","last_clean":"2023-10-07T23:23:19.181344+0000","last_became_active":"2023-10-07T23:23:19.172207+0000","last_became_peered":"2023-10-07T23:23:19.172207+0000","last_unstale":"2023-10-07T23:23:19.181344+0000","last_undegraded":"2023-10-07T23:23:19.181344+0000","last_fullsized":"2023-10-07T23:23:19.181344+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:41:16.387166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201654+0000","last_change":"2023-10-07T23:23:18.150995+0000","last_active":"2023-10-07T23:23:19.201654+0000","last_peered":"2023-10-07T23:23:19.201654+0000","last_clean":"2023-10-07T23:23:19.201654+0000","last_became_active":"2023-10-07T23:23:18.150726+0000","last_became_peered":"2023-10-07T23:23:18.150726+0000","last_unstale":"2023-10-07T23:23:19.201654+0000","last_undegraded":"2023-10-07T23:23:19.201654+0000","last_fullsized":"2023-10-07T23:23:19.201654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:37.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.177582+0000","last_change":"2023-10-07T23:23:19.173666+0000","last_active":"2023-10-07T23:23:19.177582+0000","last_peered":"2023-10-07T23:23:19.177582+0000","last_clean":"2023-10-07T23:23:19.177582+0000","last_became_active":"2023-10-07T23:23:19.173381+0000","last_became_peered":"2023-10-07T23:23:19.173381+0000","last_unstale":"2023-10-07T23:23:19.177582+0000","last_undegraded":"2023-10-07T23:23:19.177582+0000","last_fullsized":"2023-10-07T23:23:19.177582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:27.873018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199028+0000","last_change":"2023-10-07T23:23:18.152230+0000","last_active":"2023-10-07T23:23:19.199028+0000","last_peered":"2023-10-07T23:23:19.199028+0000","last_clean":"2023-10-07T23:23:19.199028+0000","last_became_active":"2023-10-07T23:23:18.151740+0000","last_became_peered":"2023-10-07T23:23:18.151740+0000","last_unstale":"2023-10-07T23:23:19.199028+0000","last_undegraded":"2023-10-07T23:23:19.199028+0000","last_fullsized":"2023-10-07T23:23:19.199028+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:56.878935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193138+0000","last_change":"2023-10-07T23:23:19.182696+0000","last_active":"2023-10-07T23:23:19.193138+0000","last_peered":"2023-10-07T23:23:19.193138+0000","last_clean":"2023-10-07T23:23:19.193138+0000","last_became_active":"2023-10-07T23:23:19.181957+0000","last_became_peered":"2023-10-07T23:23:19.181957+0000","last_unstale":"2023-10-07T23:23:19.193138+0000","last_undegraded":"2023-10-07T23:23:19.193138+0000","last_fullsized":"2023-10-07T23:23:19.193138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:58.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198389+0000","last_change":"2023-10-07T23:23:18.152280+0000","last_active":"2023-10-07T23:23:19.198389+0000","last_peered":"2023-10-07T23:23:19.198389+0000","last_clean":"2023-10-07T23:23:19.198389+0000","last_became_active":"2023-10-07T23:23:18.151434+0000","last_became_peered":"2023-10-07T23:23:18.151434+0000","last_unstale":"2023-10-07T23:23:19.198389+0000","last_undegraded":"2023-10-07T23:23:19.198389+0000","last_fullsized":"2023-10-07T23:23:19.198389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:04.549104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183402+0000","last_change":"2023-10-07T23:23:19.179167+0000","last_active":"2023-10-07T23:23:19.183402+0000","last_peered":"2023-10-07T23:23:19.183402+0000","last_clean":"2023-10-07T23:23:19.183402+0000","last_became_active":"2023-10-07T23:23:19.178614+0000","last_became_peered":"2023-10-07T23:23:19.178614+0000","last_unstale":"2023-10-07T23:23:19.183402+0000","last_undegraded":"2023-10-07T23:23:19.183402+0000","last_fullsized":"2023-10-07T23:23:19.183402+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:31.121470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.173362+0000","last_change":"2023-10-07T23:23:18.158202+0000","last_active":"2023-10-07T23:23:19.173362+0000","last_peered":"2023-10-07T23:23:19.173362+0000","last_clean":"2023-10-07T23:23:19.173362+0000","last_became_active":"2023-10-07T23:23:18.157863+0000","last_became_peered":"2023-10-07T23:23:18.157863+0000","last_unstale":"2023-10-07T23:23:19.173362+0000","last_undegraded":"2023-10-07T23:23:19.173362+0000","last_fullsized":"2023-10-07T23:23:19.173362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184937+0000","last_change":"2023-10-07T23:23:19.176870+0000","last_active":"2023-10-07T23:23:19.184937+0000","last_peered":"2023-10-07T23:23:19.184937+0000","last_clean":"2023-10-07T23:23:19.184937+0000","last_became_active":"2023-10-07T23:23:19.176579+0000","last_became_peered":"2023-10-07T23:23:19.176579+0000","last_unstale":"2023-10-07T23:23:19.184937+0000","last_undegraded":"2023-10-07T23:23:19.184937+0000","last_fullsized":"2023-10-07T23:23:19.184937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:06.023961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172517+0000","last_change":"2023-10-07T23:23:18.152141+0000","last_active":"2023-10-07T23:23:19.172517+0000","last_peered":"2023-10-07T23:23:19.172517+0000","last_clean":"2023-10-07T23:23:19.172517+0000","last_became_active":"2023-10-07T23:23:18.151575+0000","last_became_peered":"2023-10-07T23:23:18.151575+0000","last_unstale":"2023-10-07T23:23:19.172517+0000","last_undegraded":"2023-10-07T23:23:19.172517+0000","last_fullsized":"2023-10-07T23:23:19.172517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:04.549104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189503+0000","last_change":"2023-10-07T23:23:19.180726+0000","last_active":"2023-10-07T23:23:19.189503+0000","last_peered":"2023-10-07T23:23:19.189503+0000","last_clean":"2023-10-07T23:23:19.189503+0000","last_became_active":"2023-10-07T23:23:19.180423+0000","last_became_peered":"2023-10-07T23:23:19.180423+0000","last_unstale":"2023-10-07T23:23:19.189503+0000","last_undegraded":"2023-10-07T23:23:19.189503+0000","last_fullsized":"2023-10-07T23:23:19.189503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:45.694357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164521+0000","last_change":"2023-10-07T23:23:18.151584+0000","last_active":"2023-10-07T23:23:19.164521+0000","last_peered":"2023-10-07T23:23:19.164521+0000","last_clean":"2023-10-07T23:23:19.164521+0000","last_became_active":"2023-10-07T23:23:18.151258+0000","last_became_peered":"2023-10-07T23:23:18.151258+0000","last_unstale":"2023-10-07T23:23:19.164521+0000","last_undegraded":"2023-10-07T23:23:19.164521+0000","last_fullsized":"2023-10-07T23:23:19.164521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:32.091844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.185965+0000","last_change":"2023-10-07T23:23:19.181403+0000","last_active":"2023-10-07T23:23:19.185965+0000","last_peered":"2023-10-07T23:23:19.185965+0000","last_clean":"2023-10-07T23:23:19.185965+0000","last_became_active":"2023-10-07T23:23:19.181021+0000","last_became_peered":"2023-10-07T23:23:19.181021+0000","last_unstale":"2023-10-07T23:23:19.185965+0000","last_undegraded":"2023-10-07T23:23:19.185965+0000","last_fullsized":"2023-10-07T23:23:19.185965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:01:07.053652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.174684+0000","last_change":"2023-10-07T23:23:18.151555+0000","last_active":"2023-10-07T23:23:19.174684+0000","last_peered":"2023-10-07T23:23:19.174684+0000","last_clean":"2023-10-07T23:23:19.174684+0000","last_became_active":"2023-10-07T23:23:18.151263+0000","last_became_peered":"2023-10-07T23:23:18.151263+0000","last_unstale":"2023-10-07T23:23:19.174684+0000","last_undegraded":"2023-10-07T23:23:19.174684+0000","last_fullsized":"2023-10-07T23:23:19.174684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186655+0000","last_change":"2023-10-07T23:23:19.179609+0000","last_active":"2023-10-07T23:23:19.186655+0000","last_peered":"2023-10-07T23:23:19.186655+0000","last_clean":"2023-10-07T23:23:19.186655+0000","last_became_active":"2023-10-07T23:23:19.179306+0000","last_became_peered":"2023-10-07T23:23:19.179306+0000","last_unstale":"2023-10-07T23:23:19.186655+0000","last_undegraded":"2023-10-07T23:23:19.186655+0000","last_fullsized":"2023-10-07T23:23:19.186655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:19.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201206+0000","last_change":"2023-10-07T23:23:18.150283+0000","last_active":"2023-10-07T23:23:19.201206+0000","last_peered":"2023-10-07T23:23:19.201206+0000","last_clean":"2023-10-07T23:23:19.201206+0000","last_became_active":"2023-10-07T23:23:18.149788+0000","last_became_peered":"2023-10-07T23:23:18.149788+0000","last_unstale":"2023-10-07T23:23:19.201206+0000","last_undegraded":"2023-10-07T23:23:19.201206+0000","last_fullsized":"2023-10-07T23:23:19.201206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:17.169971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189397+0000","last_change":"2023-10-07T23:23:19.186320+0000","last_active":"2023-10-07T23:23:19.189397+0000","last_peered":"2023-10-07T23:23:19.189397+0000","last_clean":"2023-10-07T23:23:19.189397+0000","last_became_active":"2023-10-07T23:23:19.186005+0000","last_became_peered":"2023-10-07T23:23:19.186005+0000","last_unstale":"2023-10-07T23:23:19.189397+0000","last_undegraded":"2023-10-07T23:23:19.189397+0000","last_fullsized":"2023-10-07T23:23:19.189397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.202599+0000","last_change":"2023-10-07T23:23:18.147963+0000","last_active":"2023-10-07T23:23:19.202599+0000","last_peered":"2023-10-07T23:23:19.202599+0000","last_clean":"2023-10-07T23:23:19.202599+0000","last_became_active":"2023-10-07T23:23:18.147582+0000","last_became_peered":"2023-10-07T23:23:18.147582+0000","last_unstale":"2023-10-07T23:23:19.202599+0000","last_undegraded":"2023-10-07T23:23:19.202599+0000","last_fullsized":"2023-10-07T23:23:19.202599+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:59.641185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186454+0000","last_change":"2023-10-07T23:23:19.182488+0000","last_active":"2023-10-07T23:23:19.186454+0000","last_peered":"2023-10-07T23:23:19.186454+0000","last_clean":"2023-10-07T23:23:19.186454+0000","last_became_active":"2023-10-07T23:23:19.182209+0000","last_became_peered":"2023-10-07T23:23:19.182209+0000","last_unstale":"2023-10-07T23:23:19.186454+0000","last_undegraded":"2023-10-07T23:23:19.186454+0000","last_fullsized":"2023-10-07T23:23:19.186454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:33:55.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199822+0000","last_change":"2023-10-07T23:23:18.158468+0000","last_active":"2023-10-07T23:23:19.199822+0000","last_peered":"2023-10-07T23:23:19.199822+0000","last_clean":"2023-10-07T23:23:19.199822+0000","last_became_active":"2023-10-07T23:23:18.158108+0000","last_became_peered":"2023-10-07T23:23:18.158108+0000","last_unstale":"2023-10-07T23:23:19.199822+0000","last_undegraded":"2023-10-07T23:23:19.199822+0000","last_fullsized":"2023-10-07T23:23:19.199822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:26.855971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183506+0000","last_change":"2023-10-07T23:23:19.178918+0000","last_active":"2023-10-07T23:23:19.183506+0000","last_peered":"2023-10-07T23:23:19.183506+0000","last_clean":"2023-10-07T23:23:19.183506+0000","last_became_active":"2023-10-07T23:23:19.178555+0000","last_became_peered":"2023-10-07T23:23:19.178555+0000","last_unstale":"2023-10-07T23:23:19.183506+0000","last_undegraded":"2023-10-07T23:23:19.183506+0000","last_fullsized":"2023-10-07T23:23:19.183506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171915+0000","last_change":"2023-10-07T23:23:18.146106+0000","last_active":"2023-10-07T23:23:19.171915+0000","last_peered":"2023-10-07T23:23:19.171915+0000","last_clean":"2023-10-07T23:23:19.171915+0000","last_became_active":"2023-10-07T23:23:18.145759+0000","last_became_peered":"2023-10-07T23:23:18.145759+0000","last_unstale":"2023-10-07T23:23:19.171915+0000","last_undegraded":"2023-10-07T23:23:19.171915+0000","last_fullsized":"2023-10-07T23:23:19.171915+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186259+0000","last_change":"2023-10-07T23:23:19.178246+0000","last_active":"2023-10-07T23:23:19.186259+0000","last_peered":"2023-10-07T23:23:19.186259+0000","last_clean":"2023-10-07T23:23:19.186259+0000","last_became_active":"2023-10-07T23:23:19.177888+0000","last_became_peered":"2023-10-07T23:23:19.177888+0000","last_unstale":"2023-10-07T23:23:19.186259+0000","last_undegraded":"2023-10-07T23:23:19.186259+0000","last_fullsized":"2023-10-07T23:23:19.186259+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:31.121470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.165532+0000","last_change":"2023-10-07T23:23:18.157316+0000","last_active":"2023-10-07T23:23:19.165532+0000","last_peered":"2023-10-07T23:23:19.165532+0000","last_clean":"2023-10-07T23:23:19.165532+0000","last_became_active":"2023-10-07T23:23:18.156970+0000","last_became_peered":"2023-10-07T23:23:18.156970+0000","last_unstale":"2023-10-07T23:23:19.165532+0000","last_undegraded":"2023-10-07T23:23:19.165532+0000","last_fullsized":"2023-10-07T23:23:19.165532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:17.630732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184976+0000","last_change":"2023-10-07T23:23:19.180230+0000","last_active":"2023-10-07T23:23:19.184976+0000","last_peered":"2023-10-07T23:23:19.184976+0000","last_clean":"2023-10-07T23:23:19.184976+0000","last_became_active":"2023-10-07T23:23:19.179895+0000","last_became_peered":"2023-10-07T23:23:19.179895+0000","last_unstale":"2023-10-07T23:23:19.184976+0000","last_undegraded":"2023-10-07T23:23:19.184976+0000","last_fullsized":"2023-10-07T23:23:19.184976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.202797+0000","last_change":"2023-10-07T23:23:18.153592+0000","last_active":"2023-10-07T23:23:19.202797+0000","last_peered":"2023-10-07T23:23:19.202797+0000","last_clean":"2023-10-07T23:23:19.202797+0000","last_became_active":"2023-10-07T23:23:18.153275+0000","last_became_peered":"2023-10-07T23:23:18.153275+0000","last_unstale":"2023-10-07T23:23:19.202797+0000","last_undegraded":"2023-10-07T23:23:19.202797+0000","last_fullsized":"2023-10-07T23:23:19.202797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:37:59.169936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188659+0000","last_change":"2023-10-07T23:23:19.181274+0000","last_active":"2023-10-07T23:23:19.188659+0000","last_peered":"2023-10-07T23:23:19.188659+0000","last_clean":"2023-10-07T23:23:19.188659+0000","last_became_active":"2023-10-07T23:23:19.180318+0000","last_became_peered":"2023-10-07T23:23:19.180318+0000","last_unstale":"2023-10-07T23:23:19.188659+0000","last_undegraded":"2023-10-07T23:23:19.188659+0000","last_fullsized":"2023-10-07T23:23:19.188659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:49.869252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199215+0000","last_change":"2023-10-07T23:23:18.149907+0000","last_active":"2023-10-07T23:23:19.199215+0000","last_peered":"2023-10-07T23:23:19.199215+0000","last_clean":"2023-10-07T23:23:19.199215+0000","last_became_active":"2023-10-07T23:23:18.149496+0000","last_became_peered":"2023-10-07T23:23:18.149496+0000","last_unstale":"2023-10-07T23:23:19.199215+0000","last_undegraded":"2023-10-07T23:23:19.199215+0000","last_fullsized":"2023-10-07T23:23:19.199215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:49.516646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198250+0000","last_change":"2023-10-07T23:23:19.185237+0000","last_active":"2023-10-07T23:23:19.198250+0000","last_peered":"2023-10-07T23:23:19.198250+0000","last_clean":"2023-10-07T23:23:19.198250+0000","last_became_active":"2023-10-07T23:23:19.183842+0000","last_became_peered":"2023-10-07T23:23:19.183842+0000","last_unstale":"2023-10-07T23:23:19.198250+0000","last_undegraded":"2023-10-07T23:23:19.198250+0000","last_fullsized":"2023-10-07T23:23:19.198250+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:01:07.053652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199534+0000","last_change":"2023-10-07T23:23:18.158315+0000","last_active":"2023-10-07T23:23:19.199534+0000","last_peered":"2023-10-07T23:23:19.199534+0000","last_clean":"2023-10-07T23:23:19.199534+0000","last_became_active":"2023-10-07T23:23:18.157879+0000","last_became_peered":"2023-10-07T23:23:18.157879+0000","last_unstale":"2023-10-07T23:23:19.199534+0000","last_undegraded":"2023-10-07T23:23:19.199534+0000","last_fullsized":"2023-10-07T23:23:19.199534+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:37:59.169936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184415+0000","last_change":"2023-10-07T23:23:19.178214+0000","last_active":"2023-10-07T23:23:19.184415+0000","last_peered":"2023-10-07T23:23:19.184415+0000","last_clean":"2023-10-07T23:23:19.184415+0000","last_became_active":"2023-10-07T23:23:19.177922+0000","last_became_peered":"2023-10-07T23:23:19.177922+0000","last_unstale":"2023-10-07T23:23:19.184415+0000","last_undegraded":"2023-10-07T23:23:19.184415+0000","last_fullsized":"2023-10-07T23:23:19.184415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:18.187018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201121+0000","last_change":"2023-10-07T23:23:18.158185+0000","last_active":"2023-10-07T23:23:19.201121+0000","last_peered":"2023-10-07T23:23:19.201121+0000","last_clean":"2023-10-07T23:23:19.201121+0000","last_became_active":"2023-10-07T23:23:18.157869+0000","last_became_peered":"2023-10-07T23:23:18.157869+0000","last_unstale":"2023-10-07T23:23:19.201121+0000","last_undegraded":"2023-10-07T23:23:19.201121+0000","last_fullsized":"2023-10-07T23:23:19.201121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:37.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186430+0000","last_change":"2023-10-07T23:23:19.180287+0000","last_active":"2023-10-07T23:23:19.186430+0000","last_peered":"2023-10-07T23:23:19.186430+0000","last_clean":"2023-10-07T23:23:19.186430+0000","last_became_active":"2023-10-07T23:23:19.179961+0000","last_became_peered":"2023-10-07T23:23:19.179961+0000","last_unstale":"2023-10-07T23:23:19.186430+0000","last_undegraded":"2023-10-07T23:23:19.186430+0000","last_fullsized":"2023-10-07T23:23:19.186430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:35.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.166849+0000","last_change":"2023-10-07T23:23:18.153128+0000","last_active":"2023-10-07T23:23:19.166849+0000","last_peered":"2023-10-07T23:23:19.166849+0000","last_clean":"2023-10-07T23:23:19.166849+0000","last_became_active":"2023-10-07T23:23:18.152809+0000","last_became_peered":"2023-10-07T23:23:18.152809+0000","last_unstale":"2023-10-07T23:23:19.166849+0000","last_undegraded":"2023-10-07T23:23:19.166849+0000","last_fullsized":"2023-10-07T23:23:19.166849+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186690+0000","last_change":"2023-10-07T23:23:19.177662+0000","last_active":"2023-10-07T23:23:19.186690+0000","last_peered":"2023-10-07T23:23:19.186690+0000","last_clean":"2023-10-07T23:23:19.186690+0000","last_became_active":"2023-10-07T23:23:19.177277+0000","last_became_peered":"2023-10-07T23:23:19.177277+0000","last_unstale":"2023-10-07T23:23:19.186690+0000","last_undegraded":"2023-10-07T23:23:19.186690+0000","last_fullsized":"2023-10-07T23:23:19.186690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:40.187820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199120+0000","last_change":"2023-10-07T23:23:18.152022+0000","last_active":"2023-10-07T23:23:19.199120+0000","last_peered":"2023-10-07T23:23:19.199120+0000","last_clean":"2023-10-07T23:23:19.199120+0000","last_became_active":"2023-10-07T23:23:18.151332+0000","last_became_peered":"2023-10-07T23:23:18.151332+0000","last_unstale":"2023-10-07T23:23:19.199120+0000","last_undegraded":"2023-10-07T23:23:19.199120+0000","last_fullsized":"2023-10-07T23:23:19.199120+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188549+0000","last_change":"2023-10-07T23:23:19.180238+0000","last_active":"2023-10-07T23:23:19.188549+0000","last_peered":"2023-10-07T23:23:19.188549+0000","last_clean":"2023-10-07T23:23:19.188549+0000","last_became_active":"2023-10-07T23:23:19.179951+0000","last_became_peered":"2023-10-07T23:23:19.179951+0000","last_unstale":"2023-10-07T23:23:19.188549+0000","last_undegraded":"2023-10-07T23:23:19.188549+0000","last_fullsized":"2023-10-07T23:23:19.188549+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:59.755697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.173376+0000","last_change":"2023-10-07T23:23:18.150904+0000","last_active":"2023-10-07T23:23:19.173376+0000","last_peered":"2023-10-07T23:23:19.173376+0000","last_clean":"2023-10-07T23:23:19.173376+0000","last_became_active":"2023-10-07T23:23:18.150557+0000","last_became_peered":"2023-10-07T23:23:18.150557+0000","last_unstale":"2023-10-07T23:23:19.173376+0000","last_undegraded":"2023-10-07T23:23:19.173376+0000","last_fullsized":"2023-10-07T23:23:19.173376+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188661+0000","last_change":"2023-10-07T23:23:19.181286+0000","last_active":"2023-10-07T23:23:19.188661+0000","last_peered":"2023-10-07T23:23:19.188661+0000","last_clean":"2023-10-07T23:23:19.188661+0000","last_became_active":"2023-10-07T23:23:19.179965+0000","last_became_peered":"2023-10-07T23:23:19.179965+0000","last_unstale":"2023-10-07T23:23:19.188661+0000","last_undegraded":"2023-10-07T23:23:19.188661+0000","last_fullsized":"2023-10-07T23:23:19.188661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:35:18.621894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.169785+0000","last_change":"2023-10-07T23:23:18.154745+0000","last_active":"2023-10-07T23:23:19.169785+0000","last_peered":"2023-10-07T23:23:19.169785+0000","last_clean":"2023-10-07T23:23:19.169785+0000","last_became_active":"2023-10-07T23:23:18.154451+0000","last_became_peered":"2023-10-07T23:23:18.154451+0000","last_unstale":"2023-10-07T23:23:19.169785+0000","last_undegraded":"2023-10-07T23:23:19.169785+0000","last_fullsized":"2023-10-07T23:23:19.169785+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:03.033487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.180716+0000","last_change":"2023-10-07T23:23:19.176831+0000","last_active":"2023-10-07T23:23:19.180716+0000","last_peered":"2023-10-07T23:23:19.180716+0000","last_clean":"2023-10-07T23:23:19.180716+0000","last_became_active":"2023-10-07T23:23:19.176547+0000","last_became_peered":"2023-10-07T23:23:19.176547+0000","last_unstale":"2023-10-07T23:23:19.180716+0000","last_undegraded":"2023-10-07T23:23:19.180716+0000","last_fullsized":"2023-10-07T23:23:19.180716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171854+0000","last_change":"2023-10-07T23:23:18.146178+0000","last_active":"2023-10-07T23:23:19.171854+0000","last_peered":"2023-10-07T23:23:19.171854+0000","last_clean":"2023-10-07T23:23:19.171854+0000","last_became_active":"2023-10-07T23:23:18.145843+0000","last_became_peered":"2023-10-07T23:23:18.145843+0000","last_unstale":"2023-10-07T23:23:19.171854+0000","last_undegraded":"2023-10-07T23:23:19.171854+0000","last_fullsized":"2023-10-07T23:23:19.171854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:44.677310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186404+0000","last_change":"2023-10-07T23:23:19.181653+0000","last_active":"2023-10-07T23:23:19.186404+0000","last_peered":"2023-10-07T23:23:19.186404+0000","last_clean":"2023-10-07T23:23:19.186404+0000","last_became_active":"2023-10-07T23:23:19.181367+0000","last_became_peered":"2023-10-07T23:23:19.181367+0000","last_unstale":"2023-10-07T23:23:19.186404+0000","last_undegraded":"2023-10-07T23:23:19.186404+0000","last_fullsized":"2023-10-07T23:23:19.186404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:34:06.408599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.167551+0000","last_change":"2023-10-07T23:23:18.151770+0000","last_active":"2023-10-07T23:23:19.167551+0000","last_peered":"2023-10-07T23:23:19.167551+0000","last_clean":"2023-10-07T23:23:19.167551+0000","last_became_active":"2023-10-07T23:23:18.151464+0000","last_became_peered":"2023-10-07T23:23:18.151464+0000","last_unstale":"2023-10-07T23:23:19.167551+0000","last_undegraded":"2023-10-07T23:23:19.167551+0000","last_fullsized":"2023-10-07T23:23:19.167551+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:44.677310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183470+0000","last_change":"2023-10-07T23:23:19.179414+0000","last_active":"2023-10-07T23:23:19.183470+0000","last_peered":"2023-10-07T23:23:19.183470+0000","last_clean":"2023-10-07T23:23:19.183470+0000","last_became_active":"2023-10-07T23:23:19.178629+0000","last_became_peered":"2023-10-07T23:23:19.178629+0000","last_unstale":"2023-10-07T23:23:19.183470+0000","last_undegraded":"2023-10-07T23:23:19.183470+0000","last_fullsized":"2023-10-07T23:23:19.183470+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:55.089660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172419+0000","last_change":"2023-10-07T23:23:18.149277+0000","last_active":"2023-10-07T23:23:19.172419+0000","last_peered":"2023-10-07T23:23:19.172419+0000","last_clean":"2023-10-07T23:23:19.172419+0000","last_became_active":"2023-10-07T23:23:18.148927+0000","last_became_peered":"2023-10-07T23:23:18.148927+0000","last_unstale":"2023-10-07T23:23:19.172419+0000","last_undegraded":"2023-10-07T23:23:19.172419+0000","last_fullsized":"2023-10-07T23:23:19.172419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:05.006914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188648+0000","last_change":"2023-10-07T23:23:19.185424+0000","last_active":"2023-10-07T23:23:19.188648+0000","last_peered":"2023-10-07T23:23:19.188648+0000","last_clean":"2023-10-07T23:23:19.188648+0000","last_became_active":"2023-10-07T23:23:19.185115+0000","last_became_peered":"2023-10-07T23:23:19.185115+0000","last_unstale":"2023-10-07T23:23:19.188648+0000","last_undegraded":"2023-10-07T23:23:19.188648+0000","last_fullsized":"2023-10-07T23:23:19.188648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164381+0000","last_change":"2023-10-07T23:23:18.150176+0000","last_active":"2023-10-07T23:23:19.164381+0000","last_peered":"2023-10-07T23:23:19.164381+0000","last_clean":"2023-10-07T23:23:19.164381+0000","last_became_active":"2023-10-07T23:23:18.149864+0000","last_became_peered":"2023-10-07T23:23:18.149864+0000","last_unstale":"2023-10-07T23:23:19.164381+0000","last_undegraded":"2023-10-07T23:23:19.164381+0000","last_fullsized":"2023-10-07T23:23:19.164381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:53:03.051264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186170+0000","last_change":"2023-10-07T23:23:19.176911+0000","last_active":"2023-10-07T23:23:19.186170+0000","last_peered":"2023-10-07T23:23:19.186170+0000","last_clean":"2023-10-07T23:23:19.186170+0000","last_became_active":"2023-10-07T23:23:19.176566+0000","last_became_peered":"2023-10-07T23:23:19.176566+0000","last_unstale":"2023-10-07T23:23:19.186170+0000","last_undegraded":"2023-10-07T23:23:19.186170+0000","last_fullsized":"2023-10-07T23:23:19.186170+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:46.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172019+0000","last_change":"2023-10-07T23:23:18.150448+0000","last_active":"2023-10-07T23:23:19.172019+0000","last_peered":"2023-10-07T23:23:19.172019+0000","last_clean":"2023-10-07T23:23:19.172019+0000","last_became_active":"2023-10-07T23:23:18.150131+0000","last_became_peered":"2023-10-07T23:23:18.150131+0000","last_unstale":"2023-10-07T23:23:19.172019+0000","last_undegraded":"2023-10-07T23:23:19.172019+0000","last_fullsized":"2023-10-07T23:23:19.172019+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187462+0000","last_change":"2023-10-07T23:23:19.183776+0000","last_active":"2023-10-07T23:23:19.187462+0000","last_peered":"2023-10-07T23:23:19.187462+0000","last_clean":"2023-10-07T23:23:19.187462+0000","last_became_active":"2023-10-07T23:23:19.183138+0000","last_became_peered":"2023-10-07T23:23:19.183138+0000","last_unstale":"2023-10-07T23:23:19.187462+0000","last_undegraded":"2023-10-07T23:23:19.187462+0000","last_fullsized":"2023-10-07T23:23:19.187462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.569641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164636+0000","last_change":"2023-10-07T23:23:18.155389+0000","last_active":"2023-10-07T23:23:19.164636+0000","last_peered":"2023-10-07T23:23:19.164636+0000","last_clean":"2023-10-07T23:23:19.164636+0000","last_became_active":"2023-10-07T23:23:18.155117+0000","last_became_peered":"2023-10-07T23:23:18.155117+0000","last_unstale":"2023-10-07T23:23:19.164636+0000","last_undegraded":"2023-10-07T23:23:19.164636+0000","last_fullsized":"2023-10-07T23:23:19.164636+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188936+0000","last_change":"2023-10-07T23:23:19.178104+0000","last_active":"2023-10-07T23:23:19.188936+0000","last_peered":"2023-10-07T23:23:19.188936+0000","last_clean":"2023-10-07T23:23:19.188936+0000","last_became_active":"2023-10-07T23:23:19.177710+0000","last_became_peered":"2023-10-07T23:23:19.177710+0000","last_unstale":"2023-10-07T23:23:19.188936+0000","last_undegraded":"2023-10-07T23:23:19.188936+0000","last_fullsized":"2023-10-07T23:23:19.188936+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164305+0000","last_change":"2023-10-07T23:23:18.151915+0000","last_active":"2023-10-07T23:23:19.164305+0000","last_peered":"2023-10-07T23:23:19.164305+0000","last_clean":"2023-10-07T23:23:19.164305+0000","last_became_active":"2023-10-07T23:23:18.150464+0000","last_became_peered":"2023-10-07T23:23:18.150464+0000","last_unstale":"2023-10-07T23:23:19.164305+0000","last_undegraded":"2023-10-07T23:23:19.164305+0000","last_fullsized":"2023-10-07T23:23:19.164305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:56.494192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189694+0000","last_change":"2023-10-07T23:23:19.185381+0000","last_active":"2023-10-07T23:23:19.189694+0000","last_peered":"2023-10-07T23:23:19.189694+0000","last_clean":"2023-10-07T23:23:19.189694+0000","last_became_active":"2023-10-07T23:23:19.185032+0000","last_became_peered":"2023-10-07T23:23:19.185032+0000","last_unstale":"2023-10-07T23:23:19.189694+0000","last_undegraded":"2023-10-07T23:23:19.189694+0000","last_fullsized":"2023-10-07T23:23:19.189694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172827+0000","last_change":"2023-10-07T23:23:18.152318+0000","last_active":"2023-10-07T23:23:19.172827+0000","last_peered":"2023-10-07T23:23:19.172827+0000","last_clean":"2023-10-07T23:23:19.172827+0000","last_became_active":"2023-10-07T23:23:18.151901+0000","last_became_peered":"2023-10-07T23:23:18.151901+0000","last_unstale":"2023-10-07T23:23:19.172827+0000","last_undegraded":"2023-10-07T23:23:19.172827+0000","last_fullsized":"2023-10-07T23:23:19.172827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:06.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193223+0000","last_change":"2023-10-07T23:23:19.182780+0000","last_active":"2023-10-07T23:23:19.193223+0000","last_peered":"2023-10-07T23:23:19.193223+0000","last_clean":"2023-10-07T23:23:19.193223+0000","last_became_active":"2023-10-07T23:23:19.182388+0000","last_became_peered":"2023-10-07T23:23:19.182388+0000","last_unstale":"2023-10-07T23:23:19.193223+0000","last_undegraded":"2023-10-07T23:23:19.193223+0000","last_fullsized":"2023-10-07T23:23:19.193223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201203+0000","last_change":"2023-10-07T23:23:18.154127+0000","last_active":"2023-10-07T23:23:19.201203+0000","last_peered":"2023-10-07T23:23:19.201203+0000","last_clean":"2023-10-07T23:23:19.201203+0000","last_became_active":"2023-10-07T23:23:18.153862+0000","last_became_peered":"2023-10-07T23:23:18.153862+0000","last_unstale":"2023-10-07T23:23:19.201203+0000","last_undegraded":"2023-10-07T23:23:19.201203+0000","last_fullsized":"2023-10-07T23:23:19.201203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:17.169971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171961+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:23:19.171961+0000","last_peered":"2023-10-07T23:23:19.171961+0000","last_clean":"2023-10-07T23:23:19.171961+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:23:19.171961+0000","last_undegraded":"2023-10-07T23:23:19.171961+0000","last_fullsized":"2023-10-07T23:23:19.171961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298392,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073448,"statfs":{"total":96636764160,"available":96331210752,"internally_reserved":0,"allocated":1130496,"data_stored":801527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95199999999999996,"15min":0.94999999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.216,"5min":1.258,"15min":1.258},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.03,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3149999999999999,"5min":1.508,"15min":1.508},"last":0.93100000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.872,"15min":0.872},"min":{"1min":0.67100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.04,"5min":1.302,"15min":1.302},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94399999999999995,"15min":0.95299999999999996},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3660000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0389999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.046},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.327,"5min":1.365,"15min":1.365},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.0720000000000001,"15min":1.075},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.341,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1619999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2549999999999999,"5min":1.51,"15min":1.51},"last":0.876}]},{"osd":4,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.385,"5min":1.603,"15min":1.603},"last":1.099},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.107,"15min":1.127},"min":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2809999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.194,"15min":1.216},"min":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.905,"15min":1.905},"last":1.1539999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.2270000000000001},"min":{"1min":0.85299999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.341,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.232,"15min":1.24},"min":{"1min":0.91300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6160000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.014}]}]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":218001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.94599999999999995,"15min":0.93400000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0009999999999999,"15min":0.99399999999999999},"min":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.23,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.018}]},{"osd":1,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93799999999999994,"15min":0.94599999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.292,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.044},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.67000000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.98599999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.21,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.071},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95799999999999996,"15min":0.96599999999999997},"min":{"1min":0.749,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.145,"5min":1.484,"15min":1.484},"last":0.77900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0389999999999999,"15min":1.046},"min":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.3520000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.147,"15min":1.155},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4379999999999999,"5min":2.145,"15min":2.145},"last":1.2370000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":0.97599999999999998},"min":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.161,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.369,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.1839999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0720000000000001,"15min":1.0780000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.871,"5min":0.745,"15min":0.745},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.2050000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.177,"15min":1.1830000000000001},"min":{"1min":0.88400000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7270000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.171,"15min":1.177},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.155}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":217427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.622},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0489999999999999,"15min":1.052},"min":{"1min":0.89100000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2909999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.093}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":0.97699999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0600000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88,"15min":0.878},"min":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1299999999999999,"5min":1.296,"15min":1.296},"last":0.77300000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.94399999999999995,"15min":0.94099999999999995},"min":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.222,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2549999999999999,"5min":2.452,"15min":2.452},"last":0.83399999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.008,"15min":1.024},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2050000000000001,"5min":2.649,"15min":2.649},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2050000000000001,"5min":2.077,"15min":2.077},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.95799999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2350000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.1220000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2509999999999999,"5min":2.395,"15min":2.395},"last":1.149}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.149,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.93700000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.286,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.05}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.198,"15min":1.21},"min":{"1min":0.82099999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6100000000000001,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.2},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.292,"15min":1.3160000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.3080000000000001,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.119}]}]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":557056,"data_stored":218893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91000000000000003,"15min":0.90100000000000002},"min":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.026,"5min":1.014,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.92900000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.66200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.093,"5min":1.306,"15min":1.306},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.071}]},{"osd":2,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2290000000000001,"5min":1.952,"15min":1.952},"last":1.121},{"interface":"front","average":{"1min":1.236,"5min":1.1879999999999999,"15min":1.1799999999999999},"min":{"1min":0.93500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.5740000000000001,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.2190000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.78900000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.871},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0569999999999999,"15min":1.0489999999999999},"min":{"1min":0.81499999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97199999999999998,"15min":0.97799999999999998},"min":{"1min":0.65900000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5329999999999999,"5min":1.74,"15min":1.74},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.105,"15min":1.105},"min":{"1min":0.752,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4630000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.83299999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.026,"15min":1.02},"min":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.879,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.149}]},{"osd":7,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1679999999999999,"15min":1.1739999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.161,"15min":1.165},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.47,"5min":1.919,"15min":1.919},"last":1.353}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301928,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069912,"statfs":{"total":96636764160,"available":96327589888,"internally_reserved":0,"allocated":557056,"data_stored":217967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.873,"15min":0.89600000000000002},"min":{"1min":0.46600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0329999999999999,"5min":1.244,"15min":1.244},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.92100000000000004,"15min":0.93200000000000005},"min":{"1min":0.56000000000000005,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1799999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.84499999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0149999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.997,"15min":1.018},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.234,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.234}]},{"osd":4,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1220000000000001,"15min":1.137},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.218,"5min":2.044,"15min":2.044},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.77800000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.649},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.256,"15min":1.28},"min":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.389,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.389}]},{"osd":6,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.304,"15min":1.3320000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5069999999999999,"5min":2.802,"15min":2.802},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.323,"15min":1.3540000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4590000000000001,"5min":2.758,"15min":2.758},"last":1.4590000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2869999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.484,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.85999999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3720000000000001,"5min":2.601,"15min":2.601},"last":1.133}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301932,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069908,"statfs":{"total":96636764160,"available":96327585792,"internally_reserved":0,"allocated":561152,"data_stored":218915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.752}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.266,"15min":1.266},"last":0.5},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.501,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.546,"15min":1.546},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90800000000000003,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.288,"5min":1.49,"15min":1.49},"last":0.81299999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0449999999999999,"15min":1.046},"min":{"1min":0.77700000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.042,"5min":1.1020000000000001,"15min":1.1120000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.395,"5min":1.871,"15min":1.871},"last":1.355}]},{"osd":5,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.123,"15min":1.141},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.282,"5min":2.302,"15min":2.302},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7190000000000001,"5min":1.78,"15min":1.78},"last":1.0620000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2370000000000001,"15min":1.2390000000000001},"min":{"1min":0.94499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4790000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.452,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.391}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.2909999999999999},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4390000000000001,"5min":2.282,"15min":2.282},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.339,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.212}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1130496,"data_stored":801471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.139,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.59499999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0549999999999999,"5min":1.333,"15min":1.333},"last":0.625}]},{"osd":3,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1779999999999999,"5min":1.252,"15min":1.252},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.546,"15min":1.546},"last":0.70599999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4119999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.117},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.087,"15min":1.093},"min":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1739999999999999,"15min":1.1759999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.206,"15min":1.22},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4339999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.90900000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.573,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2669999999999999,"15min":1.2729999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5249999999999999,"5min":2.093,"15min":2.093},"last":0.88}]},{"osd":7,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.4930000000000001,"5min":2.524,"15min":2.524},"last":1.014},{"interface":"front","average":{"1min":1.206,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.91800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6519999999999999,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301920,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069920,"statfs":{"total":96636764160,"available":96327598080,"internally_reserved":0,"allocated":548864,"data_stored":217427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.59599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92100000000000004,"15min":0.91200000000000003},"min":{"1min":0.77100000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.872}]},{"osd":3,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.88700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.343,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1000000000000001,"15min":1.101},"min":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.421,"5min":1.51,"15min":1.51},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.175,"15min":1.169},"min":{"1min":0.91700000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7669999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2150000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2070000000000001,"15min":1.2010000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.577,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.96699999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.27},{"interface":"front","average":{"1min":1.349,"5min":1.286,"15min":1.276},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.0760000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:23:24.404 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-07T23:23:24.401+0000 7fc366b60700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-07T23:23:26.614 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:23:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 Processor -- start 2023-10-07T23:23:26.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 -- start start 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c44158fb0 0x7f5c440ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4406b3c0 con 0x7f5c4406b850 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4406a7b0 con 0x7f5c44158fb0 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4a07d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4406aa90 con 0x7f5c44158be0 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4907b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4887a700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c44158fb0 0x7f5c440ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4907b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:35314/0 (socket says 172.21.15.57:35314) 2023-10-07T23:23:26.728 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4907b700 1 -- 172.21.15.57:0/965749829 learned_addr learned my addr 172.21.15.57:0/965749829 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4887a700 1 -- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 msgr2=0x7f5c440ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4887a700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4887a700 1 -- 172.21.15.57:0/965749829 >> v1:172.21.15.154:6789/0 conn(0x7f5c4406b850 legacy=0x7f5c440cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4887a700 1 -- 172.21.15.57:0/965749829 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4406ad70 con 0x7f5c44158fb0 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.724+0000 7f5c4907b700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:26.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4887a700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158fb0 0x7f5c440ce6b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098e0 tx=0x7f5c380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f7020acb729eb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:26.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c43fff700 1 -- 172.21.15.57:0/965749829 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800b640 con 0x7f5c44158fb0 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c43fff700 1 -- 172.21.15.57:0/965749829 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800b7a0 con 0x7f5c44158fb0 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c43fff700 1 -- 172.21.15.57:0/965749829 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800bb40 con 0x7f5c44158fb0 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158fb0 msgr2=0x7f5c440ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158fb0 0x7f5c440ce6b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098e0 tx=0x7f5c380091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- 172.21.15.57:0/965749829 shutdown_connections 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158fb0 0x7f5c440ce6b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/965749829 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158be0 0x7f5c440ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- 172.21.15.57:0/965749829 >> 172.21.15.57:0/965749829 conn(0x7f5c44068520 msgr2=0x7f5c44068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- 172.21.15.57:0/965749829 shutdown_connections 2023-10-07T23:23:26.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- 172.21.15.57:0/965749829 wait complete. 2023-10-07T23:23:26.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 Processor -- start 2023-10-07T23:23:26.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.728+0000 7f5c4a07d700 1 -- start start 2023-10-07T23:23:26.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158be0 0x7f5c4415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 0x7f5c4415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415b6b0 con 0x7f5c4406b850 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440c7ce0 con 0x7f5c44158be0 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415c1c0 con 0x7f5c44158fb0 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37240/0 (socket says 172.21.15.57:37240) 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 -- 172.21.15.57:0/2507640955 learned_addr learned my addr 172.21.15.57:0/2507640955 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:26.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4887a700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158be0 0x7f5c4415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4987c700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 0x7f5c4415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 -- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 msgr2=0x7f5c4415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 0x7f5c4415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 -- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158be0 msgr2=0x7f5c4415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158be0 0x7f5c4415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 -- 172.21.15.57:0/2507640955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009ae0 con 0x7f5c4406b850 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4987c700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 0x7f5c4415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4907b700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c170 tx=0x7f5c3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400cb50 con 0x7f5c4406b850 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3400f040 con 0x7f5c4406b850 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c38009420 con 0x7f5c4406b850 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34014480 con 0x7f5c4406b850 2023-10-07T23:23:26.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.732+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4415b890 con 0x7f5c4406b850 2023-10-07T23:23:26.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5c340147f0 con 0x7f5c4406b850 2023-10-07T23:23:26.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c440711c0 con 0x7f5c4406b850 2023-10-07T23:23:26.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c417fa700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 0x7f5c30062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:26.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c3407cfa0 con 0x7f5c4406b850 2023-10-07T23:23:26.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c4887a700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 0x7f5c30062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:26.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.736+0000 7f5c4887a700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 0x7f5c30062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c380093f0 tx=0x7f5c38017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:26.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.744+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5c340acbe0 con 0x7f5c4406b850 2023-10-07T23:23:26.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.896+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c4415c300 con 0x7f5c30060550 2023-10-07T23:23:26.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.912+0000 7f5c417fa700 1 -- 172.21.15.57:0/2507640955 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287171 (secure 0 0 0) 0x7f5c3800b910 con 0x7f5c30060550 2023-10-07T23:23:26.920 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 msgr2=0x7f5c30062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 0x7f5c30062a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c380093f0 tx=0x7f5c38017000 comp rx=0 tx=0).stop 2023-10-07T23:23:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 msgr2=0x7f5c4414d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c170 tx=0x7f5c3400b040 comp rx=0 tx=0).stop 2023-10-07T23:23:26.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 shutdown_connections 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5c44158fb0 0x7f5c4415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5c30060550 0x7f5c30062a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c44158be0 0x7f5c4415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 --2- 172.21.15.57:0/2507640955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c4406b850 0x7f5c4414d910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 >> 172.21.15.57:0/2507640955 conn(0x7f5c44068520 msgr2=0x7f5c440cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 shutdown_connections 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:26.920+0000 7f5c4a07d700 1 -- 172.21.15.57:0/2507640955 wait complete. 2023-10-07T23:23:26.923 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:23:26.937 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2023-10-07T23:23:25.112083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400144,"kb_used_data":5456,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752574576,"statfs":{"total":773094113280,"available":770636365824,"internally_reserved":0,"allocated":5586944,"data_stored":2911628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.982115"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184880+0000","last_change":"2023-10-07T23:23:19.178860+0000","last_active":"2023-10-07T23:23:19.184880+0000","last_peered":"2023-10-07T23:23:19.184880+0000","last_clean":"2023-10-07T23:23:19.184880+0000","last_became_active":"2023-10-07T23:23:19.178526+0000","last_became_peered":"2023-10-07T23:23:19.178526+0000","last_unstale":"2023-10-07T23:23:19.184880+0000","last_undegraded":"2023-10-07T23:23:19.184880+0000","last_fullsized":"2023-10-07T23:23:19.184880+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189325+0000","last_change":"2023-10-07T23:23:19.183620+0000","last_active":"2023-10-07T23:23:19.189325+0000","last_peered":"2023-10-07T23:23:19.189325+0000","last_clean":"2023-10-07T23:23:19.189325+0000","last_became_active":"2023-10-07T23:23:19.183131+0000","last_became_peered":"2023-10-07T23:23:19.183131+0000","last_unstale":"2023-10-07T23:23:19.189325+0000","last_undegraded":"2023-10-07T23:23:19.189325+0000","last_fullsized":"2023-10-07T23:23:19.189325+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189364+0000","last_change":"2023-10-07T23:23:19.178596+0000","last_active":"2023-10-07T23:23:19.189364+0000","last_peered":"2023-10-07T23:23:19.189364+0000","last_clean":"2023-10-07T23:23:19.189364+0000","last_became_active":"2023-10-07T23:23:19.178289+0000","last_became_peered":"2023-10-07T23:23:19.178289+0000","last_unstale":"2023-10-07T23:23:19.189364+0000","last_undegraded":"2023-10-07T23:23:19.189364+0000","last_fullsized":"2023-10-07T23:23:19.189364+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198239+0000","last_change":"2023-10-07T23:23:19.185639+0000","last_active":"2023-10-07T23:23:19.198239+0000","last_peered":"2023-10-07T23:23:19.198239+0000","last_clean":"2023-10-07T23:23:19.198239+0000","last_became_active":"2023-10-07T23:23:19.185238+0000","last_became_peered":"2023-10-07T23:23:19.185238+0000","last_unstale":"2023-10-07T23:23:19.198239+0000","last_undegraded":"2023-10-07T23:23:19.198239+0000","last_fullsized":"2023-10-07T23:23:19.198239+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:04.524579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189159+0000","last_change":"2023-10-07T23:23:19.182456+0000","last_active":"2023-10-07T23:23:19.189159+0000","last_peered":"2023-10-07T23:23:19.189159+0000","last_clean":"2023-10-07T23:23:19.189159+0000","last_became_active":"2023-10-07T23:23:19.182160+0000","last_became_peered":"2023-10-07T23:23:19.182160+0000","last_unstale":"2023-10-07T23:23:19.189159+0000","last_undegraded":"2023-10-07T23:23:19.189159+0000","last_fullsized":"2023-10-07T23:23:19.189159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183591+0000","last_change":"2023-10-07T23:23:19.174227+0000","last_active":"2023-10-07T23:23:19.183591+0000","last_peered":"2023-10-07T23:23:19.183591+0000","last_clean":"2023-10-07T23:23:19.183591+0000","last_became_active":"2023-10-07T23:23:19.173826+0000","last_became_peered":"2023-10-07T23:23:19.173826+0000","last_unstale":"2023-10-07T23:23:19.183591+0000","last_undegraded":"2023-10-07T23:23:19.183591+0000","last_fullsized":"2023-10-07T23:23:19.183591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:57.476739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.194529+0000","last_change":"2023-10-07T23:23:19.190038+0000","last_active":"2023-10-07T23:23:19.194529+0000","last_peered":"2023-10-07T23:23:19.194529+0000","last_clean":"2023-10-07T23:23:19.194529+0000","last_became_active":"2023-10-07T23:23:19.189767+0000","last_became_peered":"2023-10-07T23:23:19.189767+0000","last_unstale":"2023-10-07T23:23:19.194529+0000","last_undegraded":"2023-10-07T23:23:19.194529+0000","last_fullsized":"2023-10-07T23:23:19.194529+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198287+0000","last_change":"2023-10-07T23:23:19.186078+0000","last_active":"2023-10-07T23:23:19.198287+0000","last_peered":"2023-10-07T23:23:19.198287+0000","last_clean":"2023-10-07T23:23:19.198287+0000","last_became_active":"2023-10-07T23:23:19.185764+0000","last_became_peered":"2023-10-07T23:23:19.185764+0000","last_unstale":"2023-10-07T23:23:19.198287+0000","last_undegraded":"2023-10-07T23:23:19.198287+0000","last_fullsized":"2023-10-07T23:23:19.198287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:32:59.437357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198528+0000","last_change":"2023-10-07T23:23:19.189011+0000","last_active":"2023-10-07T23:23:19.198528+0000","last_peered":"2023-10-07T23:23:19.198528+0000","last_clean":"2023-10-07T23:23:19.198528+0000","last_became_active":"2023-10-07T23:23:19.188690+0000","last_became_peered":"2023-10-07T23:23:19.188690+0000","last_unstale":"2023-10-07T23:23:19.198528+0000","last_undegraded":"2023-10-07T23:23:19.198528+0000","last_fullsized":"2023-10-07T23:23:19.198528+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198473+0000","last_change":"2023-10-07T23:23:19.186540+0000","last_active":"2023-10-07T23:23:19.198473+0000","last_peered":"2023-10-07T23:23:19.198473+0000","last_clean":"2023-10-07T23:23:19.198473+0000","last_became_active":"2023-10-07T23:23:19.184407+0000","last_became_peered":"2023-10-07T23:23:19.184407+0000","last_unstale":"2023-10-07T23:23:19.198473+0000","last_undegraded":"2023-10-07T23:23:19.198473+0000","last_fullsized":"2023-10-07T23:23:19.198473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.192965+0000","last_change":"2023-10-07T23:23:19.182230+0000","last_active":"2023-10-07T23:23:19.192965+0000","last_peered":"2023-10-07T23:23:19.192965+0000","last_clean":"2023-10-07T23:23:19.192965+0000","last_became_active":"2023-10-07T23:23:19.181888+0000","last_became_peered":"2023-10-07T23:23:19.181888+0000","last_unstale":"2023-10-07T23:23:19.192965+0000","last_undegraded":"2023-10-07T23:23:19.192965+0000","last_fullsized":"2023-10-07T23:23:19.192965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:45.694357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189044+0000","last_change":"2023-10-07T23:23:19.181495+0000","last_active":"2023-10-07T23:23:19.189044+0000","last_peered":"2023-10-07T23:23:19.189044+0000","last_clean":"2023-10-07T23:23:19.189044+0000","last_became_active":"2023-10-07T23:23:19.180814+0000","last_became_peered":"2023-10-07T23:23:19.180814+0000","last_unstale":"2023-10-07T23:23:19.189044+0000","last_undegraded":"2023-10-07T23:23:19.189044+0000","last_fullsized":"2023-10-07T23:23:19.189044+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188921+0000","last_change":"2023-10-07T23:23:19.184004+0000","last_active":"2023-10-07T23:23:19.188921+0000","last_peered":"2023-10-07T23:23:19.188921+0000","last_clean":"2023-10-07T23:23:19.188921+0000","last_became_active":"2023-10-07T23:23:19.183539+0000","last_became_peered":"2023-10-07T23:23:19.183539+0000","last_unstale":"2023-10-07T23:23:19.188921+0000","last_undegraded":"2023-10-07T23:23:19.188921+0000","last_fullsized":"2023-10-07T23:23:19.188921+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:33.705262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193170+0000","last_change":"2023-10-07T23:23:19.182254+0000","last_active":"2023-10-07T23:23:19.193170+0000","last_peered":"2023-10-07T23:23:19.193170+0000","last_clean":"2023-10-07T23:23:19.193170+0000","last_became_active":"2023-10-07T23:23:19.181917+0000","last_became_peered":"2023-10-07T23:23:19.181917+0000","last_unstale":"2023-10-07T23:23:19.193170+0000","last_undegraded":"2023-10-07T23:23:19.193170+0000","last_fullsized":"2023-10-07T23:23:19.193170+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:27.631972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186689+0000","last_change":"2023-10-07T23:23:19.177405+0000","last_active":"2023-10-07T23:23:19.186689+0000","last_peered":"2023-10-07T23:23:19.186689+0000","last_clean":"2023-10-07T23:23:19.186689+0000","last_became_active":"2023-10-07T23:23:19.177107+0000","last_became_peered":"2023-10-07T23:23:19.177107+0000","last_unstale":"2023-10-07T23:23:19.186689+0000","last_undegraded":"2023-10-07T23:23:19.186689+0000","last_fullsized":"2023-10-07T23:23:19.186689+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:00:45.765891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186358+0000","last_change":"2023-10-07T23:23:19.181580+0000","last_active":"2023-10-07T23:23:19.186358+0000","last_peered":"2023-10-07T23:23:19.186358+0000","last_clean":"2023-10-07T23:23:19.186358+0000","last_became_active":"2023-10-07T23:23:19.181173+0000","last_became_peered":"2023-10-07T23:23:19.181173+0000","last_unstale":"2023-10-07T23:23:19.186358+0000","last_undegraded":"2023-10-07T23:23:19.186358+0000","last_fullsized":"2023-10-07T23:23:19.186358+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:38.438227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189619+0000","last_change":"2023-10-07T23:23:19.182528+0000","last_active":"2023-10-07T23:23:19.189619+0000","last_peered":"2023-10-07T23:23:19.189619+0000","last_clean":"2023-10-07T23:23:19.189619+0000","last_became_active":"2023-10-07T23:23:19.182260+0000","last_became_peered":"2023-10-07T23:23:19.182260+0000","last_unstale":"2023-10-07T23:23:19.189619+0000","last_undegraded":"2023-10-07T23:23:19.189619+0000","last_fullsized":"2023-10-07T23:23:19.189619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:35.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189752+0000","last_change":"2023-10-07T23:23:19.187175+0000","last_active":"2023-10-07T23:23:19.189752+0000","last_peered":"2023-10-07T23:23:19.189752+0000","last_clean":"2023-10-07T23:23:19.189752+0000","last_became_active":"2023-10-07T23:23:19.186873+0000","last_became_peered":"2023-10-07T23:23:19.186873+0000","last_unstale":"2023-10-07T23:23:19.189752+0000","last_undegraded":"2023-10-07T23:23:19.189752+0000","last_fullsized":"2023-10-07T23:23:19.189752+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:27.631972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188810+0000","last_change":"2023-10-07T23:23:19.185113+0000","last_active":"2023-10-07T23:23:19.188810+0000","last_peered":"2023-10-07T23:23:19.188810+0000","last_clean":"2023-10-07T23:23:19.188810+0000","last_became_active":"2023-10-07T23:23:19.184516+0000","last_became_peered":"2023-10-07T23:23:19.184516+0000","last_unstale":"2023-10-07T23:23:19.188810+0000","last_undegraded":"2023-10-07T23:23:19.188810+0000","last_fullsized":"2023-10-07T23:23:19.188810+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:04:12.196334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188789+0000","last_change":"2023-10-07T23:23:19.181817+0000","last_active":"2023-10-07T23:23:19.188789+0000","last_peered":"2023-10-07T23:23:19.188789+0000","last_clean":"2023-10-07T23:23:19.188789+0000","last_became_active":"2023-10-07T23:23:19.181416+0000","last_became_peered":"2023-10-07T23:23:19.181416+0000","last_unstale":"2023-10-07T23:23:19.188789+0000","last_undegraded":"2023-10-07T23:23:19.188789+0000","last_fullsized":"2023-10-07T23:23:19.188789+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:35:11.402150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187491+0000","last_change":"2023-10-07T23:23:19.183970+0000","last_active":"2023-10-07T23:23:19.187491+0000","last_peered":"2023-10-07T23:23:19.187491+0000","last_clean":"2023-10-07T23:23:19.187491+0000","last_became_active":"2023-10-07T23:23:19.183482+0000","last_became_peered":"2023-10-07T23:23:19.183482+0000","last_unstale":"2023-10-07T23:23:19.187491+0000","last_undegraded":"2023-10-07T23:23:19.187491+0000","last_fullsized":"2023-10-07T23:23:19.187491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:41.068514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184182+0000","last_change":"2023-10-07T23:23:19.179860+0000","last_active":"2023-10-07T23:23:19.184182+0000","last_peered":"2023-10-07T23:23:19.184182+0000","last_clean":"2023-10-07T23:23:19.184182+0000","last_became_active":"2023-10-07T23:23:19.178815+0000","last_became_peered":"2023-10-07T23:23:19.178815+0000","last_unstale":"2023-10-07T23:23:19.184182+0000","last_undegraded":"2023-10-07T23:23:19.184182+0000","last_fullsized":"2023-10-07T23:23:19.184182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:01.676402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184918+0000","last_change":"2023-10-07T23:23:19.175457+0000","last_active":"2023-10-07T23:23:19.184918+0000","last_peered":"2023-10-07T23:23:19.184918+0000","last_clean":"2023-10-07T23:23:19.184918+0000","last_became_active":"2023-10-07T23:23:19.175130+0000","last_became_peered":"2023-10-07T23:23:19.175130+0000","last_unstale":"2023-10-07T23:23:19.184918+0000","last_undegraded":"2023-10-07T23:23:19.184918+0000","last_fullsized":"2023-10-07T23:23:19.184918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:05.566152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186204+0000","last_change":"2023-10-07T23:23:19.179985+0000","last_active":"2023-10-07T23:23:19.186204+0000","last_peered":"2023-10-07T23:23:19.186204+0000","last_clean":"2023-10-07T23:23:19.186204+0000","last_became_active":"2023-10-07T23:23:19.179718+0000","last_became_peered":"2023-10-07T23:23:19.179718+0000","last_unstale":"2023-10-07T23:23:19.186204+0000","last_undegraded":"2023-10-07T23:23:19.186204+0000","last_fullsized":"2023-10-07T23:23:19.186204+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186240+0000","last_change":"2023-10-07T23:23:19.179985+0000","last_active":"2023-10-07T23:23:19.186240+0000","last_peered":"2023-10-07T23:23:19.186240+0000","last_clean":"2023-10-07T23:23:19.186240+0000","last_became_active":"2023-10-07T23:23:19.179663+0000","last_became_peered":"2023-10-07T23:23:19.179663+0000","last_unstale":"2023-10-07T23:23:19.186240+0000","last_undegraded":"2023-10-07T23:23:19.186240+0000","last_fullsized":"2023-10-07T23:23:19.186240+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:10.416689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188953+0000","last_change":"2023-10-07T23:23:19.184652+0000","last_active":"2023-10-07T23:23:19.188953+0000","last_peered":"2023-10-07T23:23:19.188953+0000","last_clean":"2023-10-07T23:23:19.188953+0000","last_became_active":"2023-10-07T23:23:19.184340+0000","last_became_peered":"2023-10-07T23:23:19.184340+0000","last_unstale":"2023-10-07T23:23:19.188953+0000","last_undegraded":"2023-10-07T23:23:19.188953+0000","last_fullsized":"2023-10-07T23:23:19.188953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:04.050534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189540+0000","last_change":"2023-10-07T23:23:19.180981+0000","last_active":"2023-10-07T23:23:19.189540+0000","last_peered":"2023-10-07T23:23:19.189540+0000","last_clean":"2023-10-07T23:23:19.189540+0000","last_became_active":"2023-10-07T23:23:19.180650+0000","last_became_peered":"2023-10-07T23:23:19.180650+0000","last_unstale":"2023-10-07T23:23:19.189540+0000","last_undegraded":"2023-10-07T23:23:19.189540+0000","last_fullsized":"2023-10-07T23:23:19.189540+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:10.416689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189073+0000","last_change":"2023-10-07T23:23:19.181724+0000","last_active":"2023-10-07T23:23:19.189073+0000","last_peered":"2023-10-07T23:23:19.189073+0000","last_clean":"2023-10-07T23:23:19.189073+0000","last_became_active":"2023-10-07T23:23:19.181165+0000","last_became_peered":"2023-10-07T23:23:19.181165+0000","last_unstale":"2023-10-07T23:23:19.189073+0000","last_undegraded":"2023-10-07T23:23:19.189073+0000","last_fullsized":"2023-10-07T23:23:19.189073+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:58.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186291+0000","last_change":"2023-10-07T23:23:19.179089+0000","last_active":"2023-10-07T23:23:19.186291+0000","last_peered":"2023-10-07T23:23:19.186291+0000","last_clean":"2023-10-07T23:23:19.186291+0000","last_became_active":"2023-10-07T23:23:19.178743+0000","last_became_peered":"2023-10-07T23:23:19.178743+0000","last_unstale":"2023-10-07T23:23:19.186291+0000","last_undegraded":"2023-10-07T23:23:19.186291+0000","last_fullsized":"2023-10-07T23:23:19.186291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:21.095791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187405+0000","last_change":"2023-10-07T23:23:19.175447+0000","last_active":"2023-10-07T23:23:19.187405+0000","last_peered":"2023-10-07T23:23:19.187405+0000","last_clean":"2023-10-07T23:23:19.187405+0000","last_became_active":"2023-10-07T23:23:19.175103+0000","last_became_peered":"2023-10-07T23:23:19.175103+0000","last_unstale":"2023-10-07T23:23:19.187405+0000","last_undegraded":"2023-10-07T23:23:19.187405+0000","last_fullsized":"2023-10-07T23:23:19.187405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:57.895982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183134+0000","last_change":"2023-10-07T23:23:19.175279+0000","last_active":"2023-10-07T23:23:19.183134+0000","last_peered":"2023-10-07T23:23:19.183134+0000","last_clean":"2023-10-07T23:23:19.183134+0000","last_became_active":"2023-10-07T23:23:19.174982+0000","last_became_peered":"2023-10-07T23:23:19.174982+0000","last_unstale":"2023-10-07T23:23:19.183134+0000","last_undegraded":"2023-10-07T23:23:19.183134+0000","last_fullsized":"2023-10-07T23:23:19.183134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:54.500376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.185177+0000","last_change":"2023-10-07T23:23:19.177640+0000","last_active":"2023-10-07T23:23:19.185177+0000","last_peered":"2023-10-07T23:23:19.185177+0000","last_clean":"2023-10-07T23:23:19.185177+0000","last_became_active":"2023-10-07T23:23:19.177273+0000","last_became_peered":"2023-10-07T23:23:19.177273+0000","last_unstale":"2023-10-07T23:23:19.185177+0000","last_undegraded":"2023-10-07T23:23:19.185177+0000","last_fullsized":"2023-10-07T23:23:19.185177+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:01.148050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.194205+0000","last_change":"2023-10-07T23:23:19.184868+0000","last_active":"2023-10-07T23:23:19.194205+0000","last_peered":"2023-10-07T23:23:19.194205+0000","last_clean":"2023-10-07T23:23:19.194205+0000","last_became_active":"2023-10-07T23:23:19.184580+0000","last_became_peered":"2023-10-07T23:23:19.184580+0000","last_unstale":"2023-10-07T23:23:19.194205+0000","last_undegraded":"2023-10-07T23:23:19.194205+0000","last_fullsized":"2023-10-07T23:23:19.194205+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:41.992437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.200176+0000","last_change":"2023-10-07T23:23:18.153116+0000","last_active":"2023-10-07T23:23:19.200176+0000","last_peered":"2023-10-07T23:23:19.200176+0000","last_clean":"2023-10-07T23:23:19.200176+0000","last_became_active":"2023-10-07T23:23:18.152781+0000","last_became_peered":"2023-10-07T23:23:18.152781+0000","last_unstale":"2023-10-07T23:23:19.200176+0000","last_undegraded":"2023-10-07T23:23:19.200176+0000","last_fullsized":"2023-10-07T23:23:19.200176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:26.614925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.178325+0000","last_change":"2023-10-07T23:23:19.172880+0000","last_active":"2023-10-07T23:23:19.178325+0000","last_peered":"2023-10-07T23:23:19.178325+0000","last_clean":"2023-10-07T23:23:19.178325+0000","last_became_active":"2023-10-07T23:23:19.172236+0000","last_became_peered":"2023-10-07T23:23:19.172236+0000","last_unstale":"2023-10-07T23:23:19.178325+0000","last_undegraded":"2023-10-07T23:23:19.178325+0000","last_fullsized":"2023-10-07T23:23:19.178325+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:19.000142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199114+0000","last_change":"2023-10-07T23:23:18.153585+0000","last_active":"2023-10-07T23:23:19.199114+0000","last_peered":"2023-10-07T23:23:19.199114+0000","last_clean":"2023-10-07T23:23:19.199114+0000","last_became_active":"2023-10-07T23:23:18.153280+0000","last_became_peered":"2023-10-07T23:23:18.153280+0000","last_unstale":"2023-10-07T23:23:19.199114+0000","last_undegraded":"2023-10-07T23:23:19.199114+0000","last_fullsized":"2023-10-07T23:23:19.199114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:06.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189347+0000","last_change":"2023-10-07T23:23:19.186014+0000","last_active":"2023-10-07T23:23:19.189347+0000","last_peered":"2023-10-07T23:23:19.189347+0000","last_clean":"2023-10-07T23:23:19.189347+0000","last_became_active":"2023-10-07T23:23:19.185745+0000","last_became_peered":"2023-10-07T23:23:19.185745+0000","last_unstale":"2023-10-07T23:23:19.189347+0000","last_undegraded":"2023-10-07T23:23:19.189347+0000","last_fullsized":"2023-10-07T23:23:19.189347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:01.676402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.200479+0000","last_change":"2023-10-07T23:23:18.149433+0000","last_active":"2023-10-07T23:23:19.200479+0000","last_peered":"2023-10-07T23:23:19.200479+0000","last_clean":"2023-10-07T23:23:19.200479+0000","last_became_active":"2023-10-07T23:23:18.149086+0000","last_became_peered":"2023-10-07T23:23:18.149086+0000","last_unstale":"2023-10-07T23:23:19.200479+0000","last_undegraded":"2023-10-07T23:23:19.200479+0000","last_fullsized":"2023-10-07T23:23:19.200479+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:29:44.351494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.181344+0000","last_change":"2023-10-07T23:23:19.173457+0000","last_active":"2023-10-07T23:23:19.181344+0000","last_peered":"2023-10-07T23:23:19.181344+0000","last_clean":"2023-10-07T23:23:19.181344+0000","last_became_active":"2023-10-07T23:23:19.172207+0000","last_became_peered":"2023-10-07T23:23:19.172207+0000","last_unstale":"2023-10-07T23:23:19.181344+0000","last_undegraded":"2023-10-07T23:23:19.181344+0000","last_fullsized":"2023-10-07T23:23:19.181344+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:41:16.387166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201654+0000","last_change":"2023-10-07T23:23:18.150995+0000","last_active":"2023-10-07T23:23:19.201654+0000","last_peered":"2023-10-07T23:23:19.201654+0000","last_clean":"2023-10-07T23:23:19.201654+0000","last_became_active":"2023-10-07T23:23:18.150726+0000","last_became_peered":"2023-10-07T23:23:18.150726+0000","last_unstale":"2023-10-07T23:23:19.201654+0000","last_undegraded":"2023-10-07T23:23:19.201654+0000","last_fullsized":"2023-10-07T23:23:19.201654+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:37.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.177582+0000","last_change":"2023-10-07T23:23:19.173666+0000","last_active":"2023-10-07T23:23:19.177582+0000","last_peered":"2023-10-07T23:23:19.177582+0000","last_clean":"2023-10-07T23:23:19.177582+0000","last_became_active":"2023-10-07T23:23:19.173381+0000","last_became_peered":"2023-10-07T23:23:19.173381+0000","last_unstale":"2023-10-07T23:23:19.177582+0000","last_undegraded":"2023-10-07T23:23:19.177582+0000","last_fullsized":"2023-10-07T23:23:19.177582+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:27.873018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199028+0000","last_change":"2023-10-07T23:23:18.152230+0000","last_active":"2023-10-07T23:23:19.199028+0000","last_peered":"2023-10-07T23:23:19.199028+0000","last_clean":"2023-10-07T23:23:19.199028+0000","last_became_active":"2023-10-07T23:23:18.151740+0000","last_became_peered":"2023-10-07T23:23:18.151740+0000","last_unstale":"2023-10-07T23:23:19.199028+0000","last_undegraded":"2023-10-07T23:23:19.199028+0000","last_fullsized":"2023-10-07T23:23:19.199028+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:56.878935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193138+0000","last_change":"2023-10-07T23:23:19.182696+0000","last_active":"2023-10-07T23:23:19.193138+0000","last_peered":"2023-10-07T23:23:19.193138+0000","last_clean":"2023-10-07T23:23:19.193138+0000","last_became_active":"2023-10-07T23:23:19.181957+0000","last_became_peered":"2023-10-07T23:23:19.181957+0000","last_unstale":"2023-10-07T23:23:19.193138+0000","last_undegraded":"2023-10-07T23:23:19.193138+0000","last_fullsized":"2023-10-07T23:23:19.193138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:58.934891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198389+0000","last_change":"2023-10-07T23:23:18.152280+0000","last_active":"2023-10-07T23:23:19.198389+0000","last_peered":"2023-10-07T23:23:19.198389+0000","last_clean":"2023-10-07T23:23:19.198389+0000","last_became_active":"2023-10-07T23:23:18.151434+0000","last_became_peered":"2023-10-07T23:23:18.151434+0000","last_unstale":"2023-10-07T23:23:19.198389+0000","last_undegraded":"2023-10-07T23:23:19.198389+0000","last_fullsized":"2023-10-07T23:23:19.198389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:04.549104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183402+0000","last_change":"2023-10-07T23:23:19.179167+0000","last_active":"2023-10-07T23:23:19.183402+0000","last_peered":"2023-10-07T23:23:19.183402+0000","last_clean":"2023-10-07T23:23:19.183402+0000","last_became_active":"2023-10-07T23:23:19.178614+0000","last_became_peered":"2023-10-07T23:23:19.178614+0000","last_unstale":"2023-10-07T23:23:19.183402+0000","last_undegraded":"2023-10-07T23:23:19.183402+0000","last_fullsized":"2023-10-07T23:23:19.183402+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:31.121470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.173362+0000","last_change":"2023-10-07T23:23:18.158202+0000","last_active":"2023-10-07T23:23:19.173362+0000","last_peered":"2023-10-07T23:23:19.173362+0000","last_clean":"2023-10-07T23:23:19.173362+0000","last_became_active":"2023-10-07T23:23:18.157863+0000","last_became_peered":"2023-10-07T23:23:18.157863+0000","last_unstale":"2023-10-07T23:23:19.173362+0000","last_undegraded":"2023-10-07T23:23:19.173362+0000","last_fullsized":"2023-10-07T23:23:19.173362+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184937+0000","last_change":"2023-10-07T23:23:19.176870+0000","last_active":"2023-10-07T23:23:19.184937+0000","last_peered":"2023-10-07T23:23:19.184937+0000","last_clean":"2023-10-07T23:23:19.184937+0000","last_became_active":"2023-10-07T23:23:19.176579+0000","last_became_peered":"2023-10-07T23:23:19.176579+0000","last_unstale":"2023-10-07T23:23:19.184937+0000","last_undegraded":"2023-10-07T23:23:19.184937+0000","last_fullsized":"2023-10-07T23:23:19.184937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:06.023961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172517+0000","last_change":"2023-10-07T23:23:18.152141+0000","last_active":"2023-10-07T23:23:19.172517+0000","last_peered":"2023-10-07T23:23:19.172517+0000","last_clean":"2023-10-07T23:23:19.172517+0000","last_became_active":"2023-10-07T23:23:18.151575+0000","last_became_peered":"2023-10-07T23:23:18.151575+0000","last_unstale":"2023-10-07T23:23:19.172517+0000","last_undegraded":"2023-10-07T23:23:19.172517+0000","last_fullsized":"2023-10-07T23:23:19.172517+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:04.549104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189503+0000","last_change":"2023-10-07T23:23:19.180726+0000","last_active":"2023-10-07T23:23:19.189503+0000","last_peered":"2023-10-07T23:23:19.189503+0000","last_clean":"2023-10-07T23:23:19.189503+0000","last_became_active":"2023-10-07T23:23:19.180423+0000","last_became_peered":"2023-10-07T23:23:19.180423+0000","last_unstale":"2023-10-07T23:23:19.189503+0000","last_undegraded":"2023-10-07T23:23:19.189503+0000","last_fullsized":"2023-10-07T23:23:19.189503+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:45.694357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164521+0000","last_change":"2023-10-07T23:23:18.151584+0000","last_active":"2023-10-07T23:23:19.164521+0000","last_peered":"2023-10-07T23:23:19.164521+0000","last_clean":"2023-10-07T23:23:19.164521+0000","last_became_active":"2023-10-07T23:23:18.151258+0000","last_became_peered":"2023-10-07T23:23:18.151258+0000","last_unstale":"2023-10-07T23:23:19.164521+0000","last_undegraded":"2023-10-07T23:23:19.164521+0000","last_fullsized":"2023-10-07T23:23:19.164521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:32.091844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.185965+0000","last_change":"2023-10-07T23:23:19.181403+0000","last_active":"2023-10-07T23:23:19.185965+0000","last_peered":"2023-10-07T23:23:19.185965+0000","last_clean":"2023-10-07T23:23:19.185965+0000","last_became_active":"2023-10-07T23:23:19.181021+0000","last_became_peered":"2023-10-07T23:23:19.181021+0000","last_unstale":"2023-10-07T23:23:19.185965+0000","last_undegraded":"2023-10-07T23:23:19.185965+0000","last_fullsized":"2023-10-07T23:23:19.185965+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:01:07.053652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.174684+0000","last_change":"2023-10-07T23:23:18.151555+0000","last_active":"2023-10-07T23:23:19.174684+0000","last_peered":"2023-10-07T23:23:19.174684+0000","last_clean":"2023-10-07T23:23:19.174684+0000","last_became_active":"2023-10-07T23:23:18.151263+0000","last_became_peered":"2023-10-07T23:23:18.151263+0000","last_unstale":"2023-10-07T23:23:19.174684+0000","last_undegraded":"2023-10-07T23:23:19.174684+0000","last_fullsized":"2023-10-07T23:23:19.174684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186655+0000","last_change":"2023-10-07T23:23:19.179609+0000","last_active":"2023-10-07T23:23:19.186655+0000","last_peered":"2023-10-07T23:23:19.186655+0000","last_clean":"2023-10-07T23:23:19.186655+0000","last_became_active":"2023-10-07T23:23:19.179306+0000","last_became_peered":"2023-10-07T23:23:19.179306+0000","last_unstale":"2023-10-07T23:23:19.186655+0000","last_undegraded":"2023-10-07T23:23:19.186655+0000","last_fullsized":"2023-10-07T23:23:19.186655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:19.712273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201206+0000","last_change":"2023-10-07T23:23:18.150283+0000","last_active":"2023-10-07T23:23:19.201206+0000","last_peered":"2023-10-07T23:23:19.201206+0000","last_clean":"2023-10-07T23:23:19.201206+0000","last_became_active":"2023-10-07T23:23:18.149788+0000","last_became_peered":"2023-10-07T23:23:18.149788+0000","last_unstale":"2023-10-07T23:23:19.201206+0000","last_undegraded":"2023-10-07T23:23:19.201206+0000","last_fullsized":"2023-10-07T23:23:19.201206+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:17.169971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189397+0000","last_change":"2023-10-07T23:23:19.186320+0000","last_active":"2023-10-07T23:23:19.189397+0000","last_peered":"2023-10-07T23:23:19.189397+0000","last_clean":"2023-10-07T23:23:19.189397+0000","last_became_active":"2023-10-07T23:23:19.186005+0000","last_became_peered":"2023-10-07T23:23:19.186005+0000","last_unstale":"2023-10-07T23:23:19.189397+0000","last_undegraded":"2023-10-07T23:23:19.189397+0000","last_fullsized":"2023-10-07T23:23:19.189397+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.202599+0000","last_change":"2023-10-07T23:23:18.147963+0000","last_active":"2023-10-07T23:23:19.202599+0000","last_peered":"2023-10-07T23:23:19.202599+0000","last_clean":"2023-10-07T23:23:19.202599+0000","last_became_active":"2023-10-07T23:23:18.147582+0000","last_became_peered":"2023-10-07T23:23:18.147582+0000","last_unstale":"2023-10-07T23:23:19.202599+0000","last_undegraded":"2023-10-07T23:23:19.202599+0000","last_fullsized":"2023-10-07T23:23:19.202599+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:59.641185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186454+0000","last_change":"2023-10-07T23:23:19.182488+0000","last_active":"2023-10-07T23:23:19.186454+0000","last_peered":"2023-10-07T23:23:19.186454+0000","last_clean":"2023-10-07T23:23:19.186454+0000","last_became_active":"2023-10-07T23:23:19.182209+0000","last_became_peered":"2023-10-07T23:23:19.182209+0000","last_unstale":"2023-10-07T23:23:19.186454+0000","last_undegraded":"2023-10-07T23:23:19.186454+0000","last_fullsized":"2023-10-07T23:23:19.186454+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:33:55.262039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199822+0000","last_change":"2023-10-07T23:23:18.158468+0000","last_active":"2023-10-07T23:23:19.199822+0000","last_peered":"2023-10-07T23:23:19.199822+0000","last_clean":"2023-10-07T23:23:19.199822+0000","last_became_active":"2023-10-07T23:23:18.158108+0000","last_became_peered":"2023-10-07T23:23:18.158108+0000","last_unstale":"2023-10-07T23:23:19.199822+0000","last_undegraded":"2023-10-07T23:23:19.199822+0000","last_fullsized":"2023-10-07T23:23:19.199822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:26.855971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183506+0000","last_change":"2023-10-07T23:23:19.178918+0000","last_active":"2023-10-07T23:23:19.183506+0000","last_peered":"2023-10-07T23:23:19.183506+0000","last_clean":"2023-10-07T23:23:19.183506+0000","last_became_active":"2023-10-07T23:23:19.178555+0000","last_became_peered":"2023-10-07T23:23:19.178555+0000","last_unstale":"2023-10-07T23:23:19.183506+0000","last_undegraded":"2023-10-07T23:23:19.183506+0000","last_fullsized":"2023-10-07T23:23:19.183506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:17.262216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171915+0000","last_change":"2023-10-07T23:23:18.146106+0000","last_active":"2023-10-07T23:23:19.171915+0000","last_peered":"2023-10-07T23:23:19.171915+0000","last_clean":"2023-10-07T23:23:19.171915+0000","last_became_active":"2023-10-07T23:23:18.145759+0000","last_became_peered":"2023-10-07T23:23:18.145759+0000","last_unstale":"2023-10-07T23:23:19.171915+0000","last_undegraded":"2023-10-07T23:23:19.171915+0000","last_fullsized":"2023-10-07T23:23:19.171915+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186259+0000","last_change":"2023-10-07T23:23:19.178246+0000","last_active":"2023-10-07T23:23:19.186259+0000","last_peered":"2023-10-07T23:23:19.186259+0000","last_clean":"2023-10-07T23:23:19.186259+0000","last_became_active":"2023-10-07T23:23:19.177888+0000","last_became_peered":"2023-10-07T23:23:19.177888+0000","last_unstale":"2023-10-07T23:23:19.186259+0000","last_undegraded":"2023-10-07T23:23:19.186259+0000","last_fullsized":"2023-10-07T23:23:19.186259+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:31.121470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.165532+0000","last_change":"2023-10-07T23:23:18.157316+0000","last_active":"2023-10-07T23:23:19.165532+0000","last_peered":"2023-10-07T23:23:19.165532+0000","last_clean":"2023-10-07T23:23:19.165532+0000","last_became_active":"2023-10-07T23:23:18.156970+0000","last_became_peered":"2023-10-07T23:23:18.156970+0000","last_unstale":"2023-10-07T23:23:19.165532+0000","last_undegraded":"2023-10-07T23:23:19.165532+0000","last_fullsized":"2023-10-07T23:23:19.165532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:17.630732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184976+0000","last_change":"2023-10-07T23:23:19.180230+0000","last_active":"2023-10-07T23:23:19.184976+0000","last_peered":"2023-10-07T23:23:19.184976+0000","last_clean":"2023-10-07T23:23:19.184976+0000","last_became_active":"2023-10-07T23:23:19.179895+0000","last_became_peered":"2023-10-07T23:23:19.179895+0000","last_unstale":"2023-10-07T23:23:19.184976+0000","last_undegraded":"2023-10-07T23:23:19.184976+0000","last_fullsized":"2023-10-07T23:23:19.184976+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.202797+0000","last_change":"2023-10-07T23:23:18.153592+0000","last_active":"2023-10-07T23:23:19.202797+0000","last_peered":"2023-10-07T23:23:19.202797+0000","last_clean":"2023-10-07T23:23:19.202797+0000","last_became_active":"2023-10-07T23:23:18.153275+0000","last_became_peered":"2023-10-07T23:23:18.153275+0000","last_unstale":"2023-10-07T23:23:19.202797+0000","last_undegraded":"2023-10-07T23:23:19.202797+0000","last_fullsized":"2023-10-07T23:23:19.202797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:37:59.169936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188659+0000","last_change":"2023-10-07T23:23:19.181274+0000","last_active":"2023-10-07T23:23:19.188659+0000","last_peered":"2023-10-07T23:23:19.188659+0000","last_clean":"2023-10-07T23:23:19.188659+0000","last_became_active":"2023-10-07T23:23:19.180318+0000","last_became_peered":"2023-10-07T23:23:19.180318+0000","last_unstale":"2023-10-07T23:23:19.188659+0000","last_undegraded":"2023-10-07T23:23:19.188659+0000","last_fullsized":"2023-10-07T23:23:19.188659+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:49.869252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199215+0000","last_change":"2023-10-07T23:23:18.149907+0000","last_active":"2023-10-07T23:23:19.199215+0000","last_peered":"2023-10-07T23:23:19.199215+0000","last_clean":"2023-10-07T23:23:19.199215+0000","last_became_active":"2023-10-07T23:23:18.149496+0000","last_became_peered":"2023-10-07T23:23:18.149496+0000","last_unstale":"2023-10-07T23:23:19.199215+0000","last_undegraded":"2023-10-07T23:23:19.199215+0000","last_fullsized":"2023-10-07T23:23:19.199215+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:49.516646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.198250+0000","last_change":"2023-10-07T23:23:19.185237+0000","last_active":"2023-10-07T23:23:19.198250+0000","last_peered":"2023-10-07T23:23:19.198250+0000","last_clean":"2023-10-07T23:23:19.198250+0000","last_became_active":"2023-10-07T23:23:19.183842+0000","last_became_peered":"2023-10-07T23:23:19.183842+0000","last_unstale":"2023-10-07T23:23:19.198250+0000","last_undegraded":"2023-10-07T23:23:19.198250+0000","last_fullsized":"2023-10-07T23:23:19.198250+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:01:07.053652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199534+0000","last_change":"2023-10-07T23:23:18.158315+0000","last_active":"2023-10-07T23:23:19.199534+0000","last_peered":"2023-10-07T23:23:19.199534+0000","last_clean":"2023-10-07T23:23:19.199534+0000","last_became_active":"2023-10-07T23:23:18.157879+0000","last_became_peered":"2023-10-07T23:23:18.157879+0000","last_unstale":"2023-10-07T23:23:19.199534+0000","last_undegraded":"2023-10-07T23:23:19.199534+0000","last_fullsized":"2023-10-07T23:23:19.199534+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:37:59.169936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.184415+0000","last_change":"2023-10-07T23:23:19.178214+0000","last_active":"2023-10-07T23:23:19.184415+0000","last_peered":"2023-10-07T23:23:19.184415+0000","last_clean":"2023-10-07T23:23:19.184415+0000","last_became_active":"2023-10-07T23:23:19.177922+0000","last_became_peered":"2023-10-07T23:23:19.177922+0000","last_unstale":"2023-10-07T23:23:19.184415+0000","last_undegraded":"2023-10-07T23:23:19.184415+0000","last_fullsized":"2023-10-07T23:23:19.184415+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:18.187018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201121+0000","last_change":"2023-10-07T23:23:18.158185+0000","last_active":"2023-10-07T23:23:19.201121+0000","last_peered":"2023-10-07T23:23:19.201121+0000","last_clean":"2023-10-07T23:23:19.201121+0000","last_became_active":"2023-10-07T23:23:18.157869+0000","last_became_peered":"2023-10-07T23:23:18.157869+0000","last_unstale":"2023-10-07T23:23:19.201121+0000","last_undegraded":"2023-10-07T23:23:19.201121+0000","last_fullsized":"2023-10-07T23:23:19.201121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:37.421179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186430+0000","last_change":"2023-10-07T23:23:19.180287+0000","last_active":"2023-10-07T23:23:19.186430+0000","last_peered":"2023-10-07T23:23:19.186430+0000","last_clean":"2023-10-07T23:23:19.186430+0000","last_became_active":"2023-10-07T23:23:19.179961+0000","last_became_peered":"2023-10-07T23:23:19.179961+0000","last_unstale":"2023-10-07T23:23:19.186430+0000","last_undegraded":"2023-10-07T23:23:19.186430+0000","last_fullsized":"2023-10-07T23:23:19.186430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:35.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.166849+0000","last_change":"2023-10-07T23:23:18.153128+0000","last_active":"2023-10-07T23:23:19.166849+0000","last_peered":"2023-10-07T23:23:19.166849+0000","last_clean":"2023-10-07T23:23:19.166849+0000","last_became_active":"2023-10-07T23:23:18.152809+0000","last_became_peered":"2023-10-07T23:23:18.152809+0000","last_unstale":"2023-10-07T23:23:19.166849+0000","last_undegraded":"2023-10-07T23:23:19.166849+0000","last_fullsized":"2023-10-07T23:23:19.166849+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186690+0000","last_change":"2023-10-07T23:23:19.177662+0000","last_active":"2023-10-07T23:23:19.186690+0000","last_peered":"2023-10-07T23:23:19.186690+0000","last_clean":"2023-10-07T23:23:19.186690+0000","last_became_active":"2023-10-07T23:23:19.177277+0000","last_became_peered":"2023-10-07T23:23:19.177277+0000","last_unstale":"2023-10-07T23:23:19.186690+0000","last_undegraded":"2023-10-07T23:23:19.186690+0000","last_fullsized":"2023-10-07T23:23:19.186690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:22:40.187820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.199120+0000","last_change":"2023-10-07T23:23:18.152022+0000","last_active":"2023-10-07T23:23:19.199120+0000","last_peered":"2023-10-07T23:23:19.199120+0000","last_clean":"2023-10-07T23:23:19.199120+0000","last_became_active":"2023-10-07T23:23:18.151332+0000","last_became_peered":"2023-10-07T23:23:18.151332+0000","last_unstale":"2023-10-07T23:23:19.199120+0000","last_undegraded":"2023-10-07T23:23:19.199120+0000","last_fullsized":"2023-10-07T23:23:19.199120+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:54.072612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188549+0000","last_change":"2023-10-07T23:23:19.180238+0000","last_active":"2023-10-07T23:23:19.188549+0000","last_peered":"2023-10-07T23:23:19.188549+0000","last_clean":"2023-10-07T23:23:19.188549+0000","last_became_active":"2023-10-07T23:23:19.179951+0000","last_became_peered":"2023-10-07T23:23:19.179951+0000","last_unstale":"2023-10-07T23:23:19.188549+0000","last_undegraded":"2023-10-07T23:23:19.188549+0000","last_fullsized":"2023-10-07T23:23:19.188549+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:59.755697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.173376+0000","last_change":"2023-10-07T23:23:18.150904+0000","last_active":"2023-10-07T23:23:19.173376+0000","last_peered":"2023-10-07T23:23:19.173376+0000","last_clean":"2023-10-07T23:23:19.173376+0000","last_became_active":"2023-10-07T23:23:18.150557+0000","last_became_peered":"2023-10-07T23:23:18.150557+0000","last_unstale":"2023-10-07T23:23:19.173376+0000","last_undegraded":"2023-10-07T23:23:19.173376+0000","last_fullsized":"2023-10-07T23:23:19.173376+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188661+0000","last_change":"2023-10-07T23:23:19.181286+0000","last_active":"2023-10-07T23:23:19.188661+0000","last_peered":"2023-10-07T23:23:19.188661+0000","last_clean":"2023-10-07T23:23:19.188661+0000","last_became_active":"2023-10-07T23:23:19.179965+0000","last_became_peered":"2023-10-07T23:23:19.179965+0000","last_unstale":"2023-10-07T23:23:19.188661+0000","last_undegraded":"2023-10-07T23:23:19.188661+0000","last_fullsized":"2023-10-07T23:23:19.188661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:35:18.621894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.169785+0000","last_change":"2023-10-07T23:23:18.154745+0000","last_active":"2023-10-07T23:23:19.169785+0000","last_peered":"2023-10-07T23:23:19.169785+0000","last_clean":"2023-10-07T23:23:19.169785+0000","last_became_active":"2023-10-07T23:23:18.154451+0000","last_became_peered":"2023-10-07T23:23:18.154451+0000","last_unstale":"2023-10-07T23:23:19.169785+0000","last_undegraded":"2023-10-07T23:23:19.169785+0000","last_fullsized":"2023-10-07T23:23:19.169785+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:03.033487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.180716+0000","last_change":"2023-10-07T23:23:19.176831+0000","last_active":"2023-10-07T23:23:19.180716+0000","last_peered":"2023-10-07T23:23:19.180716+0000","last_clean":"2023-10-07T23:23:19.180716+0000","last_became_active":"2023-10-07T23:23:19.176547+0000","last_became_peered":"2023-10-07T23:23:19.176547+0000","last_unstale":"2023-10-07T23:23:19.180716+0000","last_undegraded":"2023-10-07T23:23:19.180716+0000","last_fullsized":"2023-10-07T23:23:19.180716+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:25.379836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171854+0000","last_change":"2023-10-07T23:23:18.146178+0000","last_active":"2023-10-07T23:23:19.171854+0000","last_peered":"2023-10-07T23:23:19.171854+0000","last_clean":"2023-10-07T23:23:19.171854+0000","last_became_active":"2023-10-07T23:23:18.145843+0000","last_became_peered":"2023-10-07T23:23:18.145843+0000","last_unstale":"2023-10-07T23:23:19.171854+0000","last_undegraded":"2023-10-07T23:23:19.171854+0000","last_fullsized":"2023-10-07T23:23:19.171854+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:44.677310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186404+0000","last_change":"2023-10-07T23:23:19.181653+0000","last_active":"2023-10-07T23:23:19.186404+0000","last_peered":"2023-10-07T23:23:19.186404+0000","last_clean":"2023-10-07T23:23:19.186404+0000","last_became_active":"2023-10-07T23:23:19.181367+0000","last_became_peered":"2023-10-07T23:23:19.181367+0000","last_unstale":"2023-10-07T23:23:19.186404+0000","last_undegraded":"2023-10-07T23:23:19.186404+0000","last_fullsized":"2023-10-07T23:23:19.186404+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:34:06.408599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.167551+0000","last_change":"2023-10-07T23:23:18.151770+0000","last_active":"2023-10-07T23:23:19.167551+0000","last_peered":"2023-10-07T23:23:19.167551+0000","last_clean":"2023-10-07T23:23:19.167551+0000","last_became_active":"2023-10-07T23:23:18.151464+0000","last_became_peered":"2023-10-07T23:23:18.151464+0000","last_unstale":"2023-10-07T23:23:19.167551+0000","last_undegraded":"2023-10-07T23:23:19.167551+0000","last_fullsized":"2023-10-07T23:23:19.167551+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:04:44.677310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.183470+0000","last_change":"2023-10-07T23:23:19.179414+0000","last_active":"2023-10-07T23:23:19.183470+0000","last_peered":"2023-10-07T23:23:19.183470+0000","last_clean":"2023-10-07T23:23:19.183470+0000","last_became_active":"2023-10-07T23:23:19.178629+0000","last_became_peered":"2023-10-07T23:23:19.178629+0000","last_unstale":"2023-10-07T23:23:19.183470+0000","last_undegraded":"2023-10-07T23:23:19.183470+0000","last_fullsized":"2023-10-07T23:23:19.183470+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:55.089660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172419+0000","last_change":"2023-10-07T23:23:18.149277+0000","last_active":"2023-10-07T23:23:19.172419+0000","last_peered":"2023-10-07T23:23:19.172419+0000","last_clean":"2023-10-07T23:23:19.172419+0000","last_became_active":"2023-10-07T23:23:18.148927+0000","last_became_peered":"2023-10-07T23:23:18.148927+0000","last_unstale":"2023-10-07T23:23:19.172419+0000","last_undegraded":"2023-10-07T23:23:19.172419+0000","last_fullsized":"2023-10-07T23:23:19.172419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:05.006914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188648+0000","last_change":"2023-10-07T23:23:19.185424+0000","last_active":"2023-10-07T23:23:19.188648+0000","last_peered":"2023-10-07T23:23:19.188648+0000","last_clean":"2023-10-07T23:23:19.188648+0000","last_became_active":"2023-10-07T23:23:19.185115+0000","last_became_peered":"2023-10-07T23:23:19.185115+0000","last_unstale":"2023-10-07T23:23:19.188648+0000","last_undegraded":"2023-10-07T23:23:19.188648+0000","last_fullsized":"2023-10-07T23:23:19.188648+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164381+0000","last_change":"2023-10-07T23:23:18.150176+0000","last_active":"2023-10-07T23:23:19.164381+0000","last_peered":"2023-10-07T23:23:19.164381+0000","last_clean":"2023-10-07T23:23:19.164381+0000","last_became_active":"2023-10-07T23:23:18.149864+0000","last_became_peered":"2023-10-07T23:23:18.149864+0000","last_unstale":"2023-10-07T23:23:19.164381+0000","last_undegraded":"2023-10-07T23:23:19.164381+0000","last_fullsized":"2023-10-07T23:23:19.164381+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:53:03.051264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.186170+0000","last_change":"2023-10-07T23:23:19.176911+0000","last_active":"2023-10-07T23:23:19.186170+0000","last_peered":"2023-10-07T23:23:19.186170+0000","last_clean":"2023-10-07T23:23:19.186170+0000","last_became_active":"2023-10-07T23:23:19.176566+0000","last_became_peered":"2023-10-07T23:23:19.176566+0000","last_unstale":"2023-10-07T23:23:19.186170+0000","last_undegraded":"2023-10-07T23:23:19.186170+0000","last_fullsized":"2023-10-07T23:23:19.186170+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:46.110914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172019+0000","last_change":"2023-10-07T23:23:18.150448+0000","last_active":"2023-10-07T23:23:19.172019+0000","last_peered":"2023-10-07T23:23:19.172019+0000","last_clean":"2023-10-07T23:23:19.172019+0000","last_became_active":"2023-10-07T23:23:18.150131+0000","last_became_peered":"2023-10-07T23:23:18.150131+0000","last_unstale":"2023-10-07T23:23:19.172019+0000","last_undegraded":"2023-10-07T23:23:19.172019+0000","last_fullsized":"2023-10-07T23:23:19.172019+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:24.362789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.187462+0000","last_change":"2023-10-07T23:23:19.183776+0000","last_active":"2023-10-07T23:23:19.187462+0000","last_peered":"2023-10-07T23:23:19.187462+0000","last_clean":"2023-10-07T23:23:19.187462+0000","last_became_active":"2023-10-07T23:23:19.183138+0000","last_became_peered":"2023-10-07T23:23:19.183138+0000","last_unstale":"2023-10-07T23:23:19.187462+0000","last_undegraded":"2023-10-07T23:23:19.187462+0000","last_fullsized":"2023-10-07T23:23:19.187462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:43:42.569641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164636+0000","last_change":"2023-10-07T23:23:18.155389+0000","last_active":"2023-10-07T23:23:19.164636+0000","last_peered":"2023-10-07T23:23:19.164636+0000","last_clean":"2023-10-07T23:23:19.164636+0000","last_became_active":"2023-10-07T23:23:18.155117+0000","last_became_peered":"2023-10-07T23:23:18.155117+0000","last_unstale":"2023-10-07T23:23:19.164636+0000","last_undegraded":"2023-10-07T23:23:19.164636+0000","last_fullsized":"2023-10-07T23:23:19.164636+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.188936+0000","last_change":"2023-10-07T23:23:19.178104+0000","last_active":"2023-10-07T23:23:19.188936+0000","last_peered":"2023-10-07T23:23:19.188936+0000","last_clean":"2023-10-07T23:23:19.188936+0000","last_became_active":"2023-10-07T23:23:19.177710+0000","last_became_peered":"2023-10-07T23:23:19.177710+0000","last_unstale":"2023-10-07T23:23:19.188936+0000","last_undegraded":"2023-10-07T23:23:19.188936+0000","last_fullsized":"2023-10-07T23:23:19.188936+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-08T23:49:43.738388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.164305+0000","last_change":"2023-10-07T23:23:18.151915+0000","last_active":"2023-10-07T23:23:19.164305+0000","last_peered":"2023-10-07T23:23:19.164305+0000","last_clean":"2023-10-07T23:23:19.164305+0000","last_became_active":"2023-10-07T23:23:18.150464+0000","last_became_peered":"2023-10-07T23:23:18.150464+0000","last_unstale":"2023-10-07T23:23:19.164305+0000","last_undegraded":"2023-10-07T23:23:19.164305+0000","last_fullsized":"2023-10-07T23:23:19.164305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:56.494192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.189694+0000","last_change":"2023-10-07T23:23:19.185381+0000","last_active":"2023-10-07T23:23:19.189694+0000","last_peered":"2023-10-07T23:23:19.189694+0000","last_clean":"2023-10-07T23:23:19.189694+0000","last_became_active":"2023-10-07T23:23:19.185032+0000","last_became_peered":"2023-10-07T23:23:19.185032+0000","last_unstale":"2023-10-07T23:23:19.189694+0000","last_undegraded":"2023-10-07T23:23:19.189694+0000","last_fullsized":"2023-10-07T23:23:19.189694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.647779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.172827+0000","last_change":"2023-10-07T23:23:18.152318+0000","last_active":"2023-10-07T23:23:19.172827+0000","last_peered":"2023-10-07T23:23:19.172827+0000","last_clean":"2023-10-07T23:23:19.172827+0000","last_became_active":"2023-10-07T23:23:18.151901+0000","last_became_peered":"2023-10-07T23:23:18.151901+0000","last_unstale":"2023-10-07T23:23:19.172827+0000","last_undegraded":"2023-10-07T23:23:19.172827+0000","last_fullsized":"2023-10-07T23:23:19.172827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:06.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.193223+0000","last_change":"2023-10-07T23:23:19.182780+0000","last_active":"2023-10-07T23:23:19.193223+0000","last_peered":"2023-10-07T23:23:19.193223+0000","last_clean":"2023-10-07T23:23:19.193223+0000","last_became_active":"2023-10-07T23:23:19.182388+0000","last_became_peered":"2023-10-07T23:23:19.182388+0000","last_unstale":"2023-10-07T23:23:19.193223+0000","last_undegraded":"2023-10-07T23:23:19.193223+0000","last_fullsized":"2023-10-07T23:23:19.193223+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:18.119428+0000","last_clean_scrub_stamp":"2023-10-07T23:23:18.119428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:26:57.807301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.201203+0000","last_change":"2023-10-07T23:23:18.154127+0000","last_active":"2023-10-07T23:23:19.201203+0000","last_peered":"2023-10-07T23:23:19.201203+0000","last_clean":"2023-10-07T23:23:19.201203+0000","last_became_active":"2023-10-07T23:23:18.153862+0000","last_became_peered":"2023-10-07T23:23:18.153862+0000","last_unstale":"2023-10-07T23:23:19.201203+0000","last_undegraded":"2023-10-07T23:23:19.201203+0000","last_fullsized":"2023-10-07T23:23:19.201203+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:23:17.102380+0000","last_clean_scrub_stamp":"2023-10-07T23:23:17.102380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:17.169971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-07T23:23:19.171961+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:23:19.171961+0000","last_peered":"2023-10-07T23:23:19.171961+0000","last_clean":"2023-10-07T23:23:19.171961+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:23:19.171961+0000","last_undegraded":"2023-10-07T23:23:19.171961+0000","last_fullsized":"2023-10-07T23:23:19.171961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298392,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073448,"statfs":{"total":96636764160,"available":96331210752,"internally_reserved":0,"allocated":1130496,"data_stored":801527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.95199999999999996,"15min":0.94999999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.216,"5min":1.258,"15min":1.258},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.03,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3149999999999999,"5min":1.508,"15min":1.508},"last":0.93100000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.872,"15min":0.872},"min":{"1min":0.67100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.04,"5min":1.302,"15min":1.302},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.94399999999999995,"15min":0.95299999999999996},"min":{"1min":0.60999999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3660000000000001,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0389999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.036,"15min":1.046},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.327,"5min":1.365,"15min":1.365},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.0720000000000001,"15min":1.075},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.341,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.78900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.63900000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.1619999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0329999999999999},"min":{"1min":0.82999999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2549999999999999,"5min":1.51,"15min":1.51},"last":0.876}]},{"osd":4,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.0049999999999999,"15min":1.0149999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.385,"5min":1.603,"15min":1.603},"last":1.099},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.107,"15min":1.127},"min":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.2809999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.194,"15min":1.216},"min":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.351,"5min":1.905,"15min":1.905},"last":1.1539999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:23:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.21,"15min":1.2270000000000001},"min":{"1min":0.85299999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.341,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.232,"15min":1.24},"min":{"1min":0.91300000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.6160000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.014}]}]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297828,"kb_used_data":540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074012,"statfs":{"total":96636764160,"available":96331788288,"internally_reserved":0,"allocated":552960,"data_stored":218001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.94599999999999995,"15min":0.93400000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0009999999999999,"15min":0.99399999999999999},"min":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.23,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.018}]},{"osd":1,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.93799999999999994,"15min":0.94599999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.292,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.044},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.67000000000000004,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.98599999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.21,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.071},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.95799999999999996,"15min":0.96599999999999997},"min":{"1min":0.749,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.145,"5min":1.484,"15min":1.484},"last":0.77900000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0389999999999999,"15min":1.046},"min":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.3520000000000001,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.147,"15min":1.155},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4379999999999999,"5min":2.145,"15min":2.145},"last":1.2370000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.96599999999999997,"15min":0.97599999999999998},"min":{"1min":0.65100000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.161,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0549999999999999,"15min":1.071},"min":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.369,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.1839999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0720000000000001,"15min":1.0780000000000001},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.1599999999999999,"15min":1.163},"min":{"1min":0.871,"5min":0.745,"15min":0.745},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.2050000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.177,"15min":1.1830000000000001},"min":{"1min":0.88400000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7270000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.137,"5min":1.171,"15min":1.177},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.155}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":217427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.071,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.622},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0489999999999999,"15min":1.052},"min":{"1min":0.89100000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2909999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.093}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":0.97699999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0600000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.88,"15min":0.878},"min":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1299999999999999,"5min":1.296,"15min":1.296},"last":0.77300000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.94399999999999995,"15min":0.94099999999999995},"min":{"1min":0.63800000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.222,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.038,"5min":1.1080000000000001,"15min":1.1200000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2549999999999999,"5min":2.452,"15min":2.452},"last":0.83399999999999996}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.008,"15min":1.024},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2050000000000001,"5min":2.649,"15min":2.649},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.2050000000000001,"5min":2.077,"15min":2.077},"last":0.66100000000000003}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.94299999999999995,"15min":0.95799999999999996},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.2350000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.1220000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.2509999999999999,"5min":2.395,"15min":2.395},"last":1.149}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1499999999999999,"15min":1.1799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.149,"5min":3.1840000000000002,"15min":3.1840000000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.119},"min":{"1min":0.93700000000000006,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.286,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.05}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.198,"15min":1.21},"min":{"1min":0.82099999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.6100000000000001,"5min":2.7229999999999999,"15min":2.7229999999999999},"last":1.2},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.292,"15min":1.3160000000000001},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.3080000000000001,"5min":3.3029999999999999,"15min":3.3029999999999999},"last":1.119}]}]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297832,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074008,"statfs":{"total":96636764160,"available":96331784192,"internally_reserved":0,"allocated":557056,"data_stored":218893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91000000000000003,"15min":0.90100000000000002},"min":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.026,"5min":1.014,"15min":1.012},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.92900000000000005}]},{"osd":1,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.66200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.093,"5min":1.306,"15min":1.306},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.071}]},{"osd":2,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2290000000000001,"5min":1.952,"15min":1.952},"last":1.121},{"interface":"front","average":{"1min":1.236,"5min":1.1879999999999999,"15min":1.1799999999999999},"min":{"1min":0.93500000000000005,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.5740000000000001,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":1.2190000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97299999999999998,"15min":0.96699999999999997},"min":{"1min":0.78900000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.871},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.0569999999999999,"15min":1.0489999999999999},"min":{"1min":0.81499999999999995,"5min":0.75,"15min":0.75},"max":{"1min":1.645,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.97199999999999998,"15min":0.97799999999999998},"min":{"1min":0.65900000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5329999999999999,"5min":1.74,"15min":1.74},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.109,"5min":1.105,"15min":1.105},"min":{"1min":0.752,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4630000000000001,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.83299999999999996}]},{"osd":6,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.026,"15min":1.02},"min":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.879,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.149}]},{"osd":7,"last update":"Sat Oct 7 23:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1679999999999999,"15min":1.1739999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.161,"15min":1.165},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.47,"5min":1.919,"15min":1.919},"last":1.353}]}]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301928,"kb_used_data":544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069912,"statfs":{"total":96636764160,"available":96327589888,"internally_reserved":0,"allocated":557056,"data_stored":217967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.873,"15min":0.89600000000000002},"min":{"1min":0.46600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.008,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.0329999999999999,"5min":1.244,"15min":1.244},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.92100000000000004,"15min":0.93200000000000005},"min":{"1min":0.56000000000000005,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.1799999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.84499999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.44500000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.0149999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.997,"15min":1.018},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.234,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":1.234}]},{"osd":4,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.1220000000000001,"15min":1.137},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.218,"5min":2.044,"15min":2.044},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.77800000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":1.649},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.256,"15min":1.28},"min":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.389,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.389}]},{"osd":6,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.304,"15min":1.3320000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5069999999999999,"5min":2.802,"15min":2.802},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.323,"15min":1.3540000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.4590000000000001,"5min":2.758,"15min":2.758},"last":1.4590000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2869999999999999,"15min":1.3029999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.484,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1839999999999999,"15min":1.1919999999999999},"min":{"1min":0.85999999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3720000000000001,"5min":2.601,"15min":2.601},"last":1.133}]}]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301932,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069908,"statfs":{"total":96636764160,"available":96327585792,"internally_reserved":0,"allocated":561152,"data_stored":218915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52900000000000003,"5min":0.437,"15min":0.437},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.752}]},{"osd":1,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.03,"5min":1.266,"15min":1.266},"last":0.5},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.501,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1830000000000001,"5min":1.546,"15min":1.546},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.56699999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.90800000000000003,"15min":0.90100000000000002},"min":{"1min":0.53800000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.288,"5min":1.49,"15min":1.49},"last":0.81299999999999994}]},{"osd":4,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0449999999999999,"15min":1.046},"min":{"1min":0.77700000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.042,"5min":1.1020000000000001,"15min":1.1120000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.395,"5min":1.871,"15min":1.871},"last":1.355}]},{"osd":5,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.123,"15min":1.141},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.282,"5min":2.302,"15min":2.302},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.125,"5min":1.1399999999999999,"15min":1.1419999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7190000000000001,"5min":1.78,"15min":1.78},"last":1.0620000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2370000000000001,"15min":1.2390000000000001},"min":{"1min":0.94499999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.4790000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2450000000000001,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":1.452,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.391}]},{"osd":7,"last update":"Sat Oct 7 23:23:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2789999999999999,"15min":1.2909999999999999},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.4390000000000001,"5min":2.282,"15min":2.282},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.2250000000000001,"15min":1.2350000000000001},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.339,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.212}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1130496,"data_stored":801471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.139,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.59499999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0549999999999999,"5min":1.333,"15min":1.333},"last":0.625}]},{"osd":3,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.499,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1779999999999999,"5min":1.252,"15min":1.252},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3,"5min":1.546,"15min":1.546},"last":0.70599999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.107,"15min":1.117},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4119999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.117},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.087,"15min":1.093},"min":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.595,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1739999999999999,"15min":1.1759999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.206,"15min":1.22},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.4339999999999999,"5min":2.2759999999999998,"15min":2.2759999999999998},"last":0.90900000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1919999999999999,"15min":1.202},"min":{"1min":0.751,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.573,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2669999999999999,"15min":1.2729999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.5249999999999999,"5min":2.093,"15min":2.093},"last":0.88}]},{"osd":7,"last update":"Sat Oct 7 23:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.4930000000000001,"5min":2.524,"15min":2.524},"last":1.014},{"interface":"front","average":{"1min":1.206,"5min":1.25,"15min":1.2569999999999999},"min":{"1min":0.91800000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6519999999999999,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301920,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069920,"statfs":{"total":96636764160,"available":96327598080,"internally_reserved":0,"allocated":548864,"data_stored":217427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.53400000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.59599999999999997,"5min":0.496,"15min":0.496},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72599999999999998,"15min":0.71899999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92100000000000004,"15min":0.91200000000000003},"min":{"1min":0.77100000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.872}]},{"osd":3,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.92500000000000004},"min":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.88700000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.343,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.90800000000000003}]},{"osd":5,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1000000000000001,"15min":1.101},"min":{"1min":0.85299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.421,"5min":1.51,"15min":1.51},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.175,"15min":1.169},"min":{"1min":0.91700000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.7669999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.2150000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.2070000000000001,"15min":1.2010000000000001},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2230000000000001,"15min":1.2310000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":1.577,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.96699999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:23:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.27},{"interface":"front","average":{"1min":1.349,"5min":1.286,"15min":1.276},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.0760000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:23:26.946 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-07T23:23:26.947 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-07T23:23:27.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 Processor -- start 2023-10-07T23:23:27.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 -- start start 2023-10-07T23:23:27.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0d64158ce0 0x7f0d6415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 0x7f0d640c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6406c800 con 0x7f0d641590b0 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6406cdc0 con 0x7f0d64158ce0 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d698f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640ae850 con 0x7f0d6415d080 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d690f4700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0d641590b0 0x7f0d6406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37346/0 (socket says 172.21.15.57:37346) 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d690f4700 1 -- 172.21.15.57:0/1623055700 learned_addr learned my addr 172.21.15.57:0/1623055700 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d688f3700 1 --2- 172.21.15.57:0/1623055700 >> v2:172.21.15.154:3300/0 conn(0x7f0d64158ce0 0x7f0d6415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 --2- 172.21.15.57:0/1623055700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 0x7f0d640c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d637fe700 1 -- 172.21.15.57:0/1623055700 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784911636 0 0) 0x7f0d6406c800 con 0x7f0d641590b0 2023-10-07T23:23:27.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d637fe700 1 -- 172.21.15.57:0/1623055700 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d54003570 con 0x7f0d641590b0 2023-10-07T23:23:27.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 -- 172.21.15.57:0/1623055700 >> v2:172.21.15.154:3300/0 conn(0x7f0d64158ce0 msgr2=0x7f0d6415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:27.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 --2- 172.21.15.57:0/1623055700 >> v2:172.21.15.154:3300/0 conn(0x7f0d64158ce0 0x7f0d6415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 -- 172.21.15.57:0/1623055700 >> v1:172.21.15.57:6789/0 conn(0x7f0d641590b0 legacy=0x7f0d6406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 -- 172.21.15.57:0/1623055700 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6415d600 con 0x7f0d6415d080 2023-10-07T23:23:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d63fff700 1 --2- 172.21.15.57:0/1623055700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 0x7f0d640c93c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a09255614cc67f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d637fe700 1 -- 172.21.15.57:0/1623055700 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50011630 con 0x7f0d6415d080 2023-10-07T23:23:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d637fe700 1 -- 172.21.15.57:0/1623055700 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50011790 con 0x7f0d6415d080 2023-10-07T23:23:27.071 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.064+0000 7f0d637fe700 1 -- 172.21.15.57:0/1623055700 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50011b10 con 0x7f0d6415d080 2023-10-07T23:23:27.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- 172.21.15.57:0/1623055700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 msgr2=0x7f0d640c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 --2- 172.21.15.57:0/1623055700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 0x7f0d640c93c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d500098e0 tx=0x7f0d500091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- 172.21.15.57:0/1623055700 shutdown_connections 2023-10-07T23:23:27.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 --2- 172.21.15.57:0/1623055700 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d6415d080 0x7f0d640c93c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 --2- 172.21.15.57:0/1623055700 >> v2:172.21.15.154:3300/0 conn(0x7f0d64158ce0 0x7f0d6415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- 172.21.15.57:0/1623055700 >> 172.21.15.57:0/1623055700 conn(0x7f0d64067b40 msgr2=0x7f0d640c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:27.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- 172.21.15.57:0/1623055700 shutdown_connections 2023-10-07T23:23:27.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- 172.21.15.57:0/1623055700 wait complete. 2023-10-07T23:23:27.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 Processor -- start 2023-10-07T23:23:27.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.068+0000 7f0d698f5700 1 -- start start 2023-10-07T23:23:27.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d64158ce0 0x7f0d641696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d6415d080 0x7f0d640aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6415a880 con 0x7f0d641590b0 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64159c70 con 0x7f0d6415d080 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d64159f50 con 0x7f0d64158ce0 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37250/0 (socket says 172.21.15.57:37250) 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 -- 172.21.15.57:0/3447469884 learned_addr learned my addr 172.21.15.57:0/3447469884 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:27.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d690f4700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d6415d080 0x7f0d640aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d688f3700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d64158ce0 0x7f0d641696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 -- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d64158ce0 msgr2=0x7f0d641696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d64158ce0 0x7f0d641696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 -- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d6415d080 msgr2=0x7f0d640aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d6415d080 0x7f0d640aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 -- 172.21.15.57:0/3447469884 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50009420 con 0x7f0d641590b0 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d63fff700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0d640ae990 tx=0x7f0d50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50011900 con 0x7f0d641590b0 2023-10-07T23:23:27.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50011a80 con 0x7f0d641590b0 2023-10-07T23:23:27.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6415a230 con 0x7f0d641590b0 2023-10-07T23:23:27.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5001a680 con 0x7f0d641590b0 2023-10-07T23:23:27.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.072+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6415ad10 con 0x7f0d641590b0 2023-10-07T23:23:27.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6415a9f0 con 0x7f0d641590b0 2023-10-07T23:23:27.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0d5001aab0 con 0x7f0d641590b0 2023-10-07T23:23:27.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d60ff9700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 0x7f0d48062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d5007e350 con 0x7f0d641590b0 2023-10-07T23:23:27.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d688f3700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 0x7f0d48062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.076+0000 7f0d688f3700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 0x7f0d48062a00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009d10 tx=0x7f0d580091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.084+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0d5004e8a0 con 0x7f0d641590b0 2023-10-07T23:23:27.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.312+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0d640711c0 con 0x7f0d641590b0 2023-10-07T23:23:27.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.320+0000 7f0d60ff9700 1 -- 172.21.15.57:0/3447469884 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0d50051ec0 con 0x7f0d641590b0 2023-10-07T23:23:27.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 msgr2=0x7f0d48062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 0x7f0d48062a00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009d10 tx=0x7f0d580091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 msgr2=0x7f0d64159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0d640ae990 tx=0x7f0d50017000 comp rx=0 tx=0).stop 2023-10-07T23:23:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 shutdown_connections 2023-10-07T23:23:27.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0d48060550 0x7f0d48062a00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d6415d080 0x7f0d640aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d641590b0 0x7f0d64159590 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 --2- 172.21.15.57:0/3447469884 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0d64158ce0 0x7f0d641696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 >> 172.21.15.57:0/3447469884 conn(0x7f0d64067b40 msgr2=0x7f0d640caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:27.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 shutdown_connections 2023-10-07T23:23:27.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.324+0000 7f0d698f5700 1 -- 172.21.15.57:0/3447469884 wait complete. 2023-10-07T23:23:27.330 INFO:teuthology.orchestra.run.smithi057.stderr:updated caps for client.0 2023-10-07T23:23:27.343 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:23:27.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 Processor -- start 2023-10-07T23:23:27.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 -- start start 2023-10-07T23:23:27.451 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80afc00 0x7f5ab80affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ab815aa20 0x7f5ab815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab80c9a50 con 0x7f5ab80caf20 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab80b0cf0 con 0x7f5ab815aa20 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c9c90 con 0x7f5ab80afc00 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ab815aa20 0x7f5ab815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab5d9b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f5ab80caf20 0x7f5ab80c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37356/0 (socket says 172.21.15.57:37356) 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab5d9b700 1 -- 172.21.15.57:0/3719795979 learned_addr learned my addr 172.21.15.57:0/3719795979 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:27.452 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab659c700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80afc00 0x7f5ab80affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 -- 172.21.15.57:0/3719795979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80afc00 msgr2=0x7f5ab80affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab559a700 1 -- 172.21.15.57:0/3719795979 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733225901 0 0) 0x7f5aa0001110 con 0x7f5ab80caf20 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80afc00 0x7f5ab80affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 -- 172.21.15.57:0/3719795979 >> v1:172.21.15.57:6789/0 conn(0x7f5ab80caf20 legacy=0x7f5ab80c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 -- 172.21.15.57:0/3719795979 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab80c9f70 con 0x7f5ab815aa20 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab6d9d700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab815aa20 0x7f5ab815aea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009e60 tx=0x7f5aa8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135b57b2d6484840 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab559a700 1 -- 172.21.15.57:0/3719795979 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800e070 con 0x7f5ab815aa20 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab559a700 1 -- 172.21.15.57:0/3719795979 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa800bb60 con 0x7f5ab815aa20 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.448+0000 7f5ab559a700 1 -- 172.21.15.57:0/3719795979 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8012430 con 0x7f5ab815aa20 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/3719795979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab815aa20 msgr2=0x7f5ab815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab815aa20 0x7f5ab815aea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009e60 tx=0x7f5aa8009780 comp rx=0 tx=0).stop 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/3719795979 shutdown_connections 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab815aa20 0x7f5ab815aea0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- 172.21.15.57:0/3719795979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80afc00 0x7f5ab80affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/3719795979 >> 172.21.15.57:0/3719795979 conn(0x7f5ab80692f0 msgr2=0x7f5ab80696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/3719795979 shutdown_connections 2023-10-07T23:23:27.455 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/3719795979 wait complete. 2023-10-07T23:23:27.457 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 Processor -- start 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- start start 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 0x7f5ab815ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab815aa20 0x7f5ab80c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c5b90 con 0x7f5ab80afc00 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c69b0 con 0x7f5ab815aa20 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c5da0 con 0x7f5ab80caf20 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37258/0 (socket says 172.21.15.57:37258) 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 -- 172.21.15.57:0/2554781342 learned_addr learned my addr 172.21.15.57:0/2554781342 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:27.458 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab5d9b700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 0x7f5ab815ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab6d9d700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab815aa20 0x7f5ab80c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 -- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 msgr2=0x7f5ab815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 0x7f5ab815ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 -- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab815aa20 msgr2=0x7f5ab80c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab815aa20 0x7f5ab80c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 -- 172.21.15.57:0/2554781342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8009a20 con 0x7f5ab80afc00 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab5d9b700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 0x7f5ab815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab659c700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000ec80 tx=0x7f5ab00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.459 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000c9d0 con 0x7f5ab80afc00 2023-10-07T23:23:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab000cb30 con 0x7f5ab80afc00 2023-10-07T23:23:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab80c6080 con 0x7f5ab80afc00 2023-10-07T23:23:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0013470 con 0x7f5ab80afc00 2023-10-07T23:23:27.460 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.452+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab80c6640 con 0x7f5ab80afc00 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab80711c0 con 0x7f5ab80afc00 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ab0013820 con 0x7f5ab80afc00 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5aae7fc700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 0x7f5a9c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ab007cb80 con 0x7f5ab80afc00 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5ab5d9b700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 0x7f5a9c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:27.461 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.456+0000 7f5ab5d9b700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 0x7f5a9c062a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80c94b0 tx=0x7f5aa0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:27.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.464+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ab004d170 con 0x7f5ab80afc00 2023-10-07T23:23:27.660 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.656+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ab80ca250 con 0x7f5ab80afc00 2023-10-07T23:23:27.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5aae7fc700 1 -- 172.21.15.57:0/2554781342 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f5ab0050790 con 0x7f5ab80afc00 2023-10-07T23:23:27.662 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:23:27.662 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":19,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:23:21.186753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:23:27.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 msgr2=0x7f5a9c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.664 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 0x7f5a9c062a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80c94b0 tx=0x7f5aa0009040 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 msgr2=0x7f5ab8158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000ec80 tx=0x7f5ab00091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 shutdown_connections 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5a9c060550 0x7f5a9c062a00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ab815aa20 0x7f5ab80c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ab80caf20 0x7f5ab815ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 --2- 172.21.15.57:0/2554781342 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ab80afc00 0x7f5ab8158270 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 >> 172.21.15.57:0/2554781342 conn(0x7f5ab80692f0 msgr2=0x7f5ab80c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 shutdown_connections 2023-10-07T23:23:27.665 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:27.660+0000 7f5ab759e700 1 -- 172.21.15.57:0/2554781342 wait complete. 2023-10-07T23:23:27.666 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 19 2023-10-07T23:23:28.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-07T23:23:18.503008+0000', 'modified': '2023-10-07T23:23:21.186753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5359}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5359': {'gid': 5359, 'name': 'd', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6835/320776638', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 320776638}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 320776638}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-07T23:23:28.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-07T23:23:28.079 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-07T23:23:28.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.192+0000 7f1cd55b4700 1 Processor -- start 2023-10-07T23:23:28.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.192+0000 7f1cd55b4700 1 -- start start 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1cd00cbed0 0x7f1cd006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1cd006cd40 0x7f1cd00c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd0159930 con 0x7f1cd0155a60 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd00cc340 con 0x7f1cd006cd40 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd0159c10 con 0x7f1cd00cbed0 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1cd006cd40 0x7f1cd00c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1cd006cd40 0x7f1cd00c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37268/0 (socket says 172.21.15.57:37268) 2023-10-07T23:23:28.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/4032588010 learned_addr learned my addr 172.21.15.57:0/4032588010 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/4032588010 >> v2:172.21.15.154:3300/0 conn(0x7f1cd00cbed0 0x7f1cd006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/4032588010 >> v2:172.21.15.154:3300/0 conn(0x7f1cd00cbed0 msgr2=0x7f1cd006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/4032588010 >> v2:172.21.15.154:3300/0 conn(0x7f1cd00cbed0 0x7f1cd006c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/4032588010 >> v1:172.21.15.154:6789/0 conn(0x7f1cd0155a60 legacy=0x7f1cd015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/4032588010 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0156bd0 con 0x7f1cd006cd40 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/4032588010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd006cd40 0x7f1cd00c8180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1cd006c550 tx=0x7f1cc8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4370aa0225ba7e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccdffb700 1 -- 172.21.15.57:0/4032588010 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800e070 con 0x7f1cd006cd40 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccdffb700 1 -- 172.21.15.57:0/4032588010 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800ba20 con 0x7f1cd006cd40 2023-10-07T23:23:28.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1ccdffb700 1 -- 172.21.15.57:0/4032588010 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80092e0 con 0x7f1cd006cd40 2023-10-07T23:23:28.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 -- 172.21.15.57:0/4032588010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd006cd40 msgr2=0x7f1cd00c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.196+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/4032588010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd006cd40 0x7f1cd00c8180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1cd006c550 tx=0x7f1cc8009840 comp rx=0 tx=0).stop 2023-10-07T23:23:28.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 -- 172.21.15.57:0/4032588010 shutdown_connections 2023-10-07T23:23:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/4032588010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd006cd40 0x7f1cd00c8180 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/4032588010 >> v2:172.21.15.154:3300/0 conn(0x7f1cd00cbed0 0x7f1cd006c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 -- 172.21.15.57:0/4032588010 >> 172.21.15.57:0/4032588010 conn(0x7f1cd00ccef0 msgr2=0x7f1cd00c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 -- 172.21.15.57:0/4032588010 shutdown_connections 2023-10-07T23:23:28.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 -- 172.21.15.57:0/4032588010 wait complete. 2023-10-07T23:23:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 Processor -- start 2023-10-07T23:23:28.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.200+0000 7f1cd55b4700 1 -- start start 2023-10-07T23:23:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd006cd40 0x7f1cd014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 0x7f1cd006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd016b2a0 con 0x7f1cd0155a60 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00c3530 con 0x7f1cd00cbed0 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd016a690 con 0x7f1cd006cd40 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37284/0 (socket says 172.21.15.57:37284) 2023-10-07T23:23:28.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/678454905 learned_addr learned my addr 172.21.15.57:0/678454905 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cceffd700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd006cd40 0x7f1cd014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 0x7f1cd006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 -- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd006cd40 msgr2=0x7f1cd014f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd006cd40 0x7f1cd014f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 -- 172.21.15.57:0/678454905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 msgr2=0x7f1cd006b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 -- 172.21.15.57:0/678454905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8009ae0 con 0x7f1cd00cbed0 2023-10-07T23:23:28.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:28.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 0x7f1cd006aed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400ea50 tx=0x7f1cc40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cb6ffd700 1 -- 172.21.15.57:0/678454905 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400ee90 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cb6ffd700 1 -- 172.21.15.57:0/678454905 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc400c7f0 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd016a970 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cb6ffd700 1 -- 172.21.15.57:0/678454905 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400cb90 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.204+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd016af30 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cb6ffd700 1 -- 172.21.15.57:0/678454905 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f1cc4013470 con 0x7f1cd00cbed0 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cb6ffd700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 0x7f1cbc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.211 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cb6ffd700 1 -- 172.21.15.57:0/678454905 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cc407ca00 con 0x7f1cd00cbed0 2023-10-07T23:23:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cceffd700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 0x7f1cbc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cceffd700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 0x7f1cbc062a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0003aa0 tx=0x7f1cc0000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cd785d740 1 Processor -- start 2023-10-07T23:23:28.212 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.208+0000 7f1cd785d740 1 -- start start 2023-10-07T23:23:28.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1008f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1009440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xf36f30 con 0xdd6790 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xf32ed0 con 0xd8b710 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xef90b0 con 0xd8bae0 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.216 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37292/0 (socket says 172.21.15.57:37292) 2023-10-07T23:23:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1111819981 learned_addr learned my addr 172.21.15.57:0/1111819981 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cceffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1008f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1009440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 msgr2=0x1008f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1008f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 msgr2=0x1009440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1009440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1111819981 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xdd79e0 con 0xdd6790 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80033c0 tx=0x7f1cc8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f694dd2bf43941c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1caffff700 1 -- 172.21.15.57:0/1111819981 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80092e0 con 0xdd6790 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1caffff700 1 -- 172.21.15.57:0/1111819981 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800be30 con 0xdd6790 2023-10-07T23:23:28.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1caffff700 1 -- 172.21.15.57:0/1111819981 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8007610 con 0xdd6790 2023-10-07T23:23:28.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- 172.21.15.57:0/1111819981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 msgr2=0x1009980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80033c0 tx=0x7f1cc8018000 comp rx=0 tx=0).stop 2023-10-07T23:23:28.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- 172.21.15.57:0/1111819981 shutdown_connections 2023-10-07T23:23:28.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1009980 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1009440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1111819981 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1008f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- 172.21.15.57:0/1111819981 >> 172.21.15.57:0/1111819981 conn(0xd8beb0 msgr2=0xfc5930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- 172.21.15.57:0/1111819981 shutdown_connections 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.212+0000 7f1cd785d740 1 -- 172.21.15.57:0/1111819981 wait complete. 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- start start 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 Processor -- start 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0x100d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xdd6790 0xfeecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xef8d50 con 0xd8bae0 2023-10-07T23:23:28.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xed1970 con 0xdd6790 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xee1e00 con 0xd8b710 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 10 client.0 _ll_register_callbacks cb 0xf7c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37304/0 (socket says 172.21.15.57:37304) 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 -- 172.21.15.57:0/2455672322 learned_addr learned my addr 172.21.15.57:0/2455672322 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xdd6790 0xfeecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cceffd700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0x100d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 -- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 msgr2=0x100d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0x100d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 -- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xdd6790 msgr2=0xfeecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xdd6790 0xfeecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.225 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 -- 172.21.15.57:0/2455672322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0156bd0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1cc404d110 tx=0x7f1cc407ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d959c71dc522a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc404efb0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cc4054be0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xed1cc0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc4050eb0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 10 client.5514 fetch_fsmap learned FSMap version 19 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.216+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0xf01370 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xed1cc0 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 19) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1cc4050100 con 0xd8bae0 2023-10-07T23:23:28.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 10 client.5514 fetch_fsmap finished waiting for FSMap version 19 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 10 client.5514 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 20 client.5514 populate_metadata read hostname 'smithi057' 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xff02e0 0x10186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 4 client.5514 mds_command: new command op to 5359 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 --> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xed53e0 con 0xff02e0 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cceffd700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xff02e0 0x10186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 10 client.5514 ms_handle_connect on v2:172.21.15.154:6834/320776638 2023-10-07T23:23:28.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cceffd700 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xff02e0 0x10186b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a217aa36f70e348 server_cookie=b235014d22ff8b3e in_seq=0 out_seq=0 2023-10-07T23:23:28.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 1 -- 172.21.15.57:0/2455672322 <== mds.0 v2:172.21.15.154:6834/320776638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0xed53e0 con 0xff02e0 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 10 client.5514 handle_command_reply: tid=0 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 client.5514 shutdown 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 msgr2=0x100dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1cc404d110 tx=0x7f1cc407ade0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 shutdown_connections 2023-10-07T23:23:28.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xdd6790 0xfeecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xff02e0 0x10186b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xd8bae0 0x100dad0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 --2- 172.21.15.57:0/2455672322 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0x100d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cad7fa700 0 client.5514 ms_handle_reset on v2:172.21.15.154:6834/320776638 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 >> 172.21.15.57:0/2455672322 conn(0xd8beb0 msgr2=0xfbdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 shutdown_connections 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 1 -- 172.21.15.57:0/2455672322 wait complete. 2023-10-07T23:23:28.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.220+0000 7f1cd785d740 20 client.5514 trim_cache size 0 max 16384 2023-10-07T23:23:28.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.236+0000 7f1cd785d740 1 Processor -- start 2023-10-07T23:23:28.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.236+0000 7f1cd785d740 1 -- start start 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0xfbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 0xfc0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xfef0b0 con 0xdd6790 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xf2eb50 con 0xd8bae0 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xe047c0 con 0xd8b710 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37306/0 (socket says 172.21.15.57:37306) 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/3832965057 learned_addr learned my addr 172.21.15.57:0/3832965057 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cceffd700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0xfbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 0xfc0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 msgr2=0xfbfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0xfbfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 msgr2=0xfc0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 0xfc0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/3832965057 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xee7d80 con 0xdd6790 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 0xfc0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8003440 tx=0x7f1cc801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec04e9bebd0bd0cc server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cb4ff9700 1 -- 172.21.15.57:0/3832965057 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800bb90 con 0xdd6790 2023-10-07T23:23:28.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cb4ff9700 1 -- 172.21.15.57:0/3832965057 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc802d040 con 0xdd6790 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cb4ff9700 1 -- 172.21.15.57:0/3832965057 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8007650 con 0xdd6790 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- 172.21.15.57:0/3832965057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 msgr2=0x1008770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8003440 tx=0x7f1cc801f040 comp rx=0 tx=0).stop 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- 172.21.15.57:0/3832965057 shutdown_connections 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0x1008770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8bae0 0xfc0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- 172.21.15.57:0/3832965057 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8b710 0xfbfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- 172.21.15.57:0/3832965057 >> 172.21.15.57:0/3832965057 conn(0xd8beb0 msgr2=0xfbf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- 172.21.15.57:0/3832965057 shutdown_connections 2023-10-07T23:23:28.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- 172.21.15.57:0/3832965057 wait complete. 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- start start 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 Processor -- start 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1062880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.240+0000 7f1cd785d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0xf06040 con 0xdd6790 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0xee1c90 con 0xd8b710 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0xdd6ea0 con 0xd8bae0 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 10 client.0 _ll_register_callbacks cb 0xd579d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cce7fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cceffd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37322/0 (socket says 172.21.15.57:37322) 2023-10-07T23:23:28.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1124764250 learned_addr learned my addr 172.21.15.57:0/1124764250 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 msgr2=0x1062dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1062dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 msgr2=0x1062880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1062880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 -- 172.21.15.57:0/1124764250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdd79e0 con 0xdd6790 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cce7fc700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1ccf7fe700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800b740 tx=0x7f1cc802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d3fdfdec7e046 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032650 con 0xdd6790 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1cc8038030 con 0xdd6790 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cc800be30 con 0xdd6790 2023-10-07T23:23:28.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc80329c0 con 0xdd6790 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 10 client.5520 fetch_fsmap learned FSMap version 19 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0xedc130 con 0xdd6790 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc800be30 con 0xdd6790 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 19) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1cc8030070 con 0xdd6790 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 10 client.5520 fetch_fsmap finished waiting for FSMap version 19 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 10 client.5520 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 20 client.5520 populate_metadata read hostname 'smithi057' 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xfc2a00 0xfc2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 4 client.5520 mds_command: new command op to 5359 tid=0[{"prefix": "status"}] 2023-10-07T23:23:28.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 --> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] -- command(tid 0: {"prefix": "status"}) v1 -- 0xf07410 con 0xfc2a00 2023-10-07T23:23:28.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.244+0000 7f1cceffd700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xfc2a00 0xfc2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cad7fa700 10 client.5520 ms_handle_connect on v2:172.21.15.154:6834/320776638 2023-10-07T23:23:28.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cceffd700 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xfc2a00 0xfc2dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded01a7a65b7cbf3 server_cookie=35f0313980a233f in_seq=0 out_seq=0 2023-10-07T23:23:28.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cad7fa700 1 -- 172.21.15.57:0/1124764250 <== mds.0 v2:172.21.15.154:6834/320776638 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0xf07410 con 0xfc2a00 2023-10-07T23:23:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cad7fa700 10 client.5520 handle_command_reply: tid=0 2023-10-07T23:23:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 client.5520 shutdown 2023-10-07T23:23:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 msgr2=0xfc24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800b740 tx=0x7f1cc802e000 comp rx=0 tx=0).stop 2023-10-07T23:23:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 shutdown_connections 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:6834/320776638,v1:172.21.15.154:6835/320776638] conn(0xfc2a00 0xfc2dd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cad7fa700 0 client.5520 ms_handle_reset on v2:172.21.15.154:6834/320776638 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0xdd6790 0xfc24c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0xd8bae0 0x1062dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 --2- 172.21.15.57:0/1124764250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0xd8b710 0x1062880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 >> 172.21.15.57:0/1124764250 conn(0xd8beb0 msgr2=0xfec780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 shutdown_connections 2023-10-07T23:23:28.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 1 -- 172.21.15.57:0/1124764250 wait complete. 2023-10-07T23:23:28.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd785d740 20 client.5520 trim_cache size 0 max 16384 2023-10-07T23:23:28.257 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:23:28.258 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "aaf09e93-912b-470e-9df7-bb9a606f50e4", 2023-10-07T23:23:28.258 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2023-10-07T23:23:28.263 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5359, 2023-10-07T23:23:28.263 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2023-10-07T23:23:28.263 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2023-10-07T23:23:28.263 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2023-10-07T23:23:28.263 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 9.0710146619999996, 2023-10-07T23:23:28.264 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 19, 2023-10-07T23:23:28.264 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 47, 2023-10-07T23:23:28.264 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 47, 2023-10-07T23:23:28.264 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 14.290284745999999 2023-10-07T23:23:28.265 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 msgr2=0x7f1cbc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 0x7f1cbc062a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0003aa0 tx=0x7f1cc0000bc0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 msgr2=0x7f1cd006aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 0x7f1cd006aed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400ea50 tx=0x7f1cc40091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 shutdown_connections 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd0155a60 0x7f1cd006b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1cbc060550 0x7f1cbc062a00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cd00cbed0 0x7f1cd006aed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.266 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 --2- 172.21.15.57:0/678454905 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1cd006cd40 0x7f1cd014f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 >> 172.21.15.57:0/678454905 conn(0x7f1cd00ccef0 msgr2=0x7f1cd01562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 shutdown_connections 2023-10-07T23:23:28.267 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.248+0000 7f1cd55b4700 1 -- 172.21.15.57:0/678454905 wait complete. 2023-10-07T23:23:28.268 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:23:28.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 Processor -- start 2023-10-07T23:23:28.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 -- start start 2023-10-07T23:23:28.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b00c83b0 con 0x7f52b0159130 2023-10-07T23:23:28.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b00c8690 con 0x7f52b0158d60 2023-10-07T23:23:28.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.372+0000 7f52b69cd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00c81d0 con 0x7f52b006bc90 2023-10-07T23:23:28.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b61cc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f52b0159130 0x7f52b00c7ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37362/0 (socket says 172.21.15.57:37362) 2023-10-07T23:23:28.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b61cc700 1 -- 172.21.15.57:0/4007136048 learned_addr learned my addr 172.21.15.57:0/4007136048 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:28.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b51ca700 1 --2- 172.21.15.57:0/4007136048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607605156 0 0) 0x7f52b00c83b0 con 0x7f52b0159130 2023-10-07T23:23:28.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f529c003570 con 0x7f52b0159130 2023-10-07T23:23:28.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066919052 0 0) 0x7f52b00c8690 con 0x7f52b0158d60 2023-10-07T23:23:28.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b51ca700 1 -- 172.21.15.57:0/4007136048 >> v1:172.21.15.154:6789/0 conn(0x7f52b0158d60 legacy=0x7f52b006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b51ca700 1 -- 172.21.15.57:0/4007136048 >> v1:172.21.15.57:6789/0 conn(0x7f52b0159130 legacy=0x7f52b00c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b51ca700 1 -- 172.21.15.57:0/4007136048 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b00c8970 con 0x7f52b006bc90 2023-10-07T23:23:28.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b51ca700 1 --2- 172.21.15.57:0/4007136048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b00c7580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009910 tx=0x7f52a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5321d4275758e5ca server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000f6b0 con 0x7f52b006bc90 2023-10-07T23:23:28.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a000f810 con 0x7f52b006bc90 2023-10-07T23:23:28.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b49c9700 1 -- 172.21.15.57:0/4007136048 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000fbd0 con 0x7f52b006bc90 2023-10-07T23:23:28.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- 172.21.15.57:0/4007136048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 msgr2=0x7f52b00c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 --2- 172.21.15.57:0/4007136048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b00c7580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009910 tx=0x7f52a00091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- 172.21.15.57:0/4007136048 shutdown_connections 2023-10-07T23:23:28.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 --2- 172.21.15.57:0/4007136048 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b00c7580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- 172.21.15.57:0/4007136048 >> 172.21.15.57:0/4007136048 conn(0x7f52b00670e0 msgr2=0x7f52b00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- 172.21.15.57:0/4007136048 shutdown_connections 2023-10-07T23:23:28.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- 172.21.15.57:0/4007136048 wait complete. 2023-10-07T23:23:28.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 Processor -- start 2023-10-07T23:23:28.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.376+0000 7f52b69cd700 1 -- start start 2023-10-07T23:23:28.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 0x7f52b00c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00c5550 con 0x7f52b0159130 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b00c5830 con 0x7f52b0158d60 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0169730 con 0x7f52b006bc90 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b61cc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b59cb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 0x7f52b00c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b59cb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:35430/0 (socket says 172.21.15.57:35430) 2023-10-07T23:23:28.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b61cc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37328/0 (socket says 172.21.15.57:37328) 2023-10-07T23:23:28.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b59cb700 1 -- 172.21.15.57:0/1253120177 learned_addr learned my addr 172.21.15.57:0/1253120177 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 -- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 msgr2=0x7f52b0169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 -- 172.21.15.57:0/1253120177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 msgr2=0x7f52b00c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 -- 172.21.15.57:0/1253120177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a0009420 con 0x7f52b0158d60 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b59cb700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b61cc700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:23:28.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b51ca700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 0x7f52b00c3010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52a0018000 tx=0x7f52a0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000f980 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a000fb00 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00106c0 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b0169a10 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.380+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b01693e0 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b0169c50 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f52a0010a30 con 0x7f52b0158d60 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52a5ffb700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 0x7f5294062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:28.389 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52a0014070 con 0x7f52b0158d60 2023-10-07T23:23:28.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52b59cb700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 0x7f5294062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:28.390 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.384+0000 7f52b59cb700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 0x7f5294062a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52ac007fd0 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:28.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.392+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f52a004e510 con 0x7f52b0158d60 2023-10-07T23:23:28.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.596+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52b0169c50 con 0x7f52b0158d60 2023-10-07T23:23:28.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.596+0000 7f52a5ffb700 1 -- 172.21.15.57:0/1253120177 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+73 (secure 0 0 0) 0x7f52b0169c50 con 0x7f52b0158d60 2023-10-07T23:23:28.601 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:23:28.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 msgr2=0x7f5294062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 0x7f5294062a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52ac007fd0 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2023-10-07T23:23:28.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 msgr2=0x7f52b00c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:28.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 0x7f52b00c3010 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f52a0018000 tx=0x7f52a0019000 comp rx=0 tx=0).stop 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 shutdown_connections 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f52b0159130 0x7f52b00c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5294060550 0x7f5294062a00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52b0158d60 0x7f52b00c3010 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 --2- 172.21.15.57:0/1253120177 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f52b006bc90 0x7f52b0169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 >> 172.21.15.57:0/1253120177 conn(0x7f52b00670e0 msgr2=0x7f52b0155500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 shutdown_connections 2023-10-07T23:23:28.604 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:28.600+0000 7f52b69cd700 1 -- 172.21.15.57:0/1253120177 wait complete. 2023-10-07T23:23:28.615 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:23:28.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:23:28.616 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:28.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.616 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-07T23:23:28.621 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:23:28.621 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.622 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42427sec preferred_lft 42427sec 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:28.623 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 02:42:22:80:74:34 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:23:28.624 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:28.625 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:23:28.625 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.625 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::88e7:63ff:fe49:2a21/64 scope link 2023-10-07T23:23:28.625 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 5a:76:86:e5:a8:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::5876:86ff:fee5:a805/64 scope link 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:28.626 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T23:23:28.627 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::c75:b1ff:fef6:552a/64 scope link 2023-10-07T23:23:28.627 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:28.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.627 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:23:28.641 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-07T23:23:28.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.643 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-07T23:23:28.695 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:23:28.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T23:23:28.696 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:23:28.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.738 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:23:28.743 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T23:23:28.744 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:23:28.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:28.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:28.744 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-07T23:23:28.761 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T23:23:28.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:28.762 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:28.812 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:28.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:28.812 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:28.819 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:28.832 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:28.833 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:28.848 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:28.849 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:28.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:28.850 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:28.900 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:23:28.901 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T23:23:28.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:28.923 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:28.924 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:28.981 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:28.982 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:28.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:28.983 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:28.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:23:28.984+0000 7f8dcc811380 -1 init, newargv = 0x55c043d85f40 newargc=15 2023-10-07T23:23:28.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43566]: starting ceph client 2023-10-07T23:23:28.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43566]: starting fuse 2023-10-07T23:23:30.092 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:30.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:30.110 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:30.169 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:30.169 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:30.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:30.170 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:30.178 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:23:30.178 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:23:30.178 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:23:30.178 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:23:30.179 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:30.180 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-10-07T23:23:30.181 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:23:30.261 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.43566.asok 2023-10-07T23:23:30.446 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43566.asok 2023-10-07T23:23:30.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:30.446 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43566.asok status 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-07T23:23:30.586 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "43566", 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5532, 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5532 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:23:30.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2868512567 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2868512567 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5532 172.21.15.57:0/2868512567", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "172.21.15.57:0/2868512567", 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-07T23:23:30.588 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 19, 2023-10-07T23:23:30.589 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 47, 2023-10-07T23:23:30.589 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:23:30.589 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-07T23:23:30.589 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-07T23:23:30.589 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:23:30.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:30.596 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-07T23:23:30.597 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:30.605 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-07T23:23:30.606 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:30.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:30.606 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:30.622 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:23:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 Processor -- start 2023-10-07T23:23:30.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 -- start start 2023-10-07T23:23:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7e780aefb0 0x7f7e7806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e7815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7815ccb0 con 0x7f7e780af380 2023-10-07T23:23:30.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7806d1c0 con 0x7f7e780aefb0 2023-10-07T23:23:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e7de3c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7815cf70 con 0x7f7e7806cd40 2023-10-07T23:23:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7e780aefb0 0x7f7e7806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7e780aefb0 0x7f7e7806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37330/0 (socket says 172.21.15.57:37330) 2023-10-07T23:23:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 -- 172.21.15.57:0/3629162527 learned_addr learned my addr 172.21.15.57:0/3629162527 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:30.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e77fff700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e7815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 -- 172.21.15.57:0/3629162527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 msgr2=0x7f7e7815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e7815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 -- 172.21.15.57:0/3629162527 >> v1:172.21.15.154:6789/0 conn(0x7f7e780af380 legacy=0x7f7e781599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 -- 172.21.15.57:0/3629162527 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7815d250 con 0x7f7e780aefb0 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e777fe700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780aefb0 0x7f7e7806c800 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e680099b0 tx=0x7f7e68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6774cbd2ae702 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e76ffd700 1 -- 172.21.15.57:0/3629162527 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68009ea0 con 0x7f7e780aefb0 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e76ffd700 1 -- 172.21.15.57:0/3629162527 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6800b860 con 0x7f7e780aefb0 2023-10-07T23:23:30.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.784+0000 7f7e76ffd700 1 -- 172.21.15.57:0/3629162527 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800bc00 con 0x7f7e780aefb0 2023-10-07T23:23:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3629162527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780aefb0 msgr2=0x7f7e7806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780aefb0 0x7f7e7806c800 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e680099b0 tx=0x7f7e68009250 comp rx=0 tx=0).stop 2023-10-07T23:23:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3629162527 shutdown_connections 2023-10-07T23:23:30.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e7815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3629162527 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780aefb0 0x7f7e7806c800 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3629162527 >> 172.21.15.57:0/3629162527 conn(0x7f7e78068630 msgr2=0x7f7e78068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3629162527 shutdown_connections 2023-10-07T23:23:30.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3629162527 wait complete. 2023-10-07T23:23:30.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 Processor -- start 2023-10-07T23:23:30.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.788+0000 7f7e7de3c700 1 -- start start 2023-10-07T23:23:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e78154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780aefb0 0x7f7e780af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78158130 con 0x7f7e780af380 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78154e40 con 0x7f7e780aefb0 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78157b20 con 0x7f7e7806cd40 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37344/0 (socket says 172.21.15.57:37344) 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 -- 172.21.15.57:0/3102495635 learned_addr learned my addr 172.21.15.57:0/3102495635 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e777fe700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e78154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e6effd700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780aefb0 0x7f7e780af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 -- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 msgr2=0x7f7e78154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:30.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e78154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 -- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780aefb0 msgr2=0x7f7e780af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780aefb0 0x7f7e780af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 -- 172.21.15.57:0/3102495635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680094f0 con 0x7f7e780af380 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e77fff700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009ec0 tx=0x7f7e7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000cb70 con 0x7f7e780af380 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7000f040 con 0x7f7e780af380 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70014480 con 0x7f7e780af380 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e780b5970 con 0x7f7e780af380 2023-10-07T23:23:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.792+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e780b5c50 con 0x7f7e780af380 2023-10-07T23:23:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e781582e0 con 0x7f7e780af380 2023-10-07T23:23:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7e70014830 con 0x7f7e780af380 2023-10-07T23:23:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e6ffff700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 0x7f7e58062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e7007d330 con 0x7f7e780af380 2023-10-07T23:23:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e777fe700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 0x7f7e58062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.796+0000 7f7e777fe700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 0x7f7e58062a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e680099b0 tx=0x7f7e68017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:30.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:30.804+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7e781582e0 con 0x7f7e780af380 2023-10-07T23:23:31.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.000+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7e781582e0 con 0x7f7e780af380 2023-10-07T23:23:31.005 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.000+0000 7f7e6ffff700 1 -- 172.21.15.57:0/3102495635 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e781582e0 con 0x7f7e780af380 2023-10-07T23:23:31.005 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:23:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 msgr2=0x7f7e58062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 0x7f7e58062a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e680099b0 tx=0x7f7e68017000 comp rx=0 tx=0).stop 2023-10-07T23:23:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 msgr2=0x7f7e781585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009ec0 tx=0x7f7e7000b040 comp rx=0 tx=0).stop 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 shutdown_connections 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e780af380 0x7f7e781585c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7e58060580 0x7f7e58062a30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780aefb0 0x7f7e780af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 --2- 172.21.15.57:0/3102495635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7e7806cd40 0x7f7e78154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 >> 172.21.15.57:0/3102495635 conn(0x7f7e78068630 msgr2=0x7f7e7815e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 shutdown_connections 2023-10-07T23:23:31.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:31.004+0000 7f7e7de3c700 1 -- 172.21.15.57:0/3102495635 wait complete. 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:23:31.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:23:31.020 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:23:31.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.020 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42410sec preferred_lft 42410sec 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-10-07T23:23:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 02:42:9b:82:8d:1f brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::4a5:81ff:fe8f:3c66/64 scope link 2023-10-07T23:23:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 76:ca:ea:ca:a1:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::74ca:eaff:feca:a10e/64 scope link 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether de:fc:19:2e:9b:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::dcfc:19ff:fe2e:9bcb/64 scope link 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:31.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.033 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:23:31.086 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-10-07T23:23:31.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.088 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-07T23:23:31.097 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:23:31.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-07T23:23:31.098 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-10-07T23:23:31.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.135 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:23:31.143 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T23:23:31.143 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T23:23:31.143 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:23:31.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:31.144 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-10-07T23:23:31.204 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-07T23:23:31.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:31.204 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:31.211 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-07T23:23:31.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:31.212 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:31.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:23:31.233 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:31.233 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:31.291 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:31.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:31.292 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:23:31.296 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:23:31.297 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-07T23:23:31.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:23:31.314 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:31.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:31.325 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:31.325 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:31.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:31.326 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:23:31.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:2023-10-07T23:23:31.377+0000 7f521b225380 -1 init, newargv = 0x56043ca09c90 newargc=15 2023-10-07T23:23:31.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39796]: starting ceph client 2023-10-07T23:23:31.389 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39796]: starting fuse 2023-10-07T23:23:32.434 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-10-07T23:23:32.451 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:32.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:32.461 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:32.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:32.462 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-10-07T23:23:32.469 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-10-07T23:23:32.469 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:23:32.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:23:32.469 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-10-07T23:23:32.470 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.1")) 2023-10-07T23:23:32.471 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-10-07T23:23:32.599 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.1.39796.asok 2023-10-07T23:23:32.785 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39796.asok 2023-10-07T23:23:32.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:32.785 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39796.asok status 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "1", 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-10-07T23:23:32.882 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "39796", 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5554, 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5554 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:23:32.883 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1106855412 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1106855412 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.5554 192.168.0.1:0/1106855412", 2023-10-07T23:23:32.884 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/1106855412", 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 19, 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 47, 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-10-07T23:23:32.885 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-10-07T23:23:32.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:32.891 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-10-07T23:23:32.891 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:32.897 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-10-07T23:23:32.897 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:32.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:32.897 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-07T23:23:32.956 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set client client_inject_fixed_oldest_tid true 2023-10-07T23:23:33.079 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 Processor -- start 2023-10-07T23:23:33.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 -- start start 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc06cf10 0x7ff8fc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff8fc0ca600 0x7ff8fc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc06a900 con 0x7ff8fc159030 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0b0450 con 0x7ff8fc0ca600 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff902822700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc06a600 con 0x7ff8fc06cf10 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff900dbf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff8fc0ca600 0x7ff8fc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff900dbf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff8fc0ca600 0x7ff8fc0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37354/0 (socket says 172.21.15.57:37354) 2023-10-07T23:23:33.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff900dbf700 1 -- 172.21.15.57:0/3876531682 learned_addr learned my addr 172.21.15.57:0/3876531682 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:33.082 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.076+0000 7ff8fb7fe700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc06cf10 0x7ff8fc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff900dbf700 1 -- 172.21.15.57:0/3876531682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc06cf10 msgr2=0x7ff8fc0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff900dbf700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc06cf10 0x7ff8fc0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff900dbf700 1 -- 172.21.15.57:0/3876531682 >> v1:172.21.15.154:6789/0 conn(0x7ff8fc159030 legacy=0x7ff8fc06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff900dbf700 1 -- 172.21.15.57:0/3876531682 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0cb350 con 0x7ff8fc0ca600 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff900dbf700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc0ca600 0x7ff8fc0caa80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009f20 tx=0x7ff8f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac8b886547442a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.083 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff8faffd700 1 -- 172.21.15.57:0/3876531682 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400e070 con 0x7ff8fc0ca600 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff8faffd700 1 -- 172.21.15.57:0/3876531682 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f400b9d0 con 0x7ff8fc0ca600 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff8faffd700 1 -- 172.21.15.57:0/3876531682 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40092e0 con 0x7ff8fc0ca600 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- 172.21.15.57:0/3876531682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc0ca600 msgr2=0x7ff8fc0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc0ca600 0x7ff8fc0caa80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009f20 tx=0x7ff8f4009840 comp rx=0 tx=0).stop 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- 172.21.15.57:0/3876531682 shutdown_connections 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc0ca600 0x7ff8fc0caa80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 --2- 172.21.15.57:0/3876531682 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc06cf10 0x7ff8fc0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- 172.21.15.57:0/3876531682 >> 172.21.15.57:0/3876531682 conn(0x7ff8fc068650 msgr2=0x7ff8fc068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- 172.21.15.57:0/3876531682 shutdown_connections 2023-10-07T23:23:33.084 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- 172.21.15.57:0/3876531682 wait complete. 2023-10-07T23:23:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 Processor -- start 2023-10-07T23:23:33.085 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.080+0000 7ff902822700 1 -- start start 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc0ca600 0x7ff8fc15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc159030 0x7ff8fc15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc159760 con 0x7ff8fc06cf10 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15be00 con 0x7ff8fc159030 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15b1f0 con 0x7ff8fc0ca600 2023-10-07T23:23:33.087 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37360/0 (socket says 172.21.15.57:37360) 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 -- 172.21.15.57:0/3193692446 learned_addr learned my addr 172.21.15.57:0/3193692446 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fb7fe700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc0ca600 0x7ff8fc15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff900dbf700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc159030 0x7ff8fc15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 -- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc0ca600 msgr2=0x7ff8fc15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc0ca600 0x7ff8fc15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 -- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc159030 msgr2=0x7ff8fc15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc159030 0x7ff8fc15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 -- 172.21.15.57:0/3193692446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4009ae0 con 0x7ff8fc06cf10 2023-10-07T23:23:33.088 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8fbfff700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c4e0 tx=0x7ff8f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0011590 con 0x7ff8fc06cf10 2023-10-07T23:23:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0011710 con 0x7ff8fc06cf10 2023-10-07T23:23:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0011ad0 con 0x7ff8fc06cf10 2023-10-07T23:23:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc15b4d0 con 0x7ff8fc06cf10 2023-10-07T23:23:33.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.084+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc15ba90 con 0x7ff8fc06cf10 2023-10-07T23:23:33.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff8f001b3b0 con 0x7ff8fc06cf10 2023-10-07T23:23:33.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc0647b0 con 0x7ff8fc06cf10 2023-10-07T23:23:33.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff8e3fff700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 0x7ff8e8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff8f007bb10 con 0x7ff8fc06cf10 2023-10-07T23:23:33.091 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff8fb7fe700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 0x7ff8e8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.088+0000 7ff8fb7fe700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 0x7ff8e8062a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0c6af0 tx=0x7ff8ec0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.096+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff8f004c100 con 0x7ff8fc06cf10 2023-10-07T23:23:33.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.260+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=client_inject_fixed_oldest_tid}] v 0) v1 -- 0x7ff8fc0cb630 con 0x7ff8fc06cf10 2023-10-07T23:23:33.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.268+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7ff8f0001150 con 0x7ff8fc06cf10 2023-10-07T23:23:33.272 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.268+0000 7ff8e3fff700 1 -- 172.21.15.57:0/3193692446 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_inject_fixed_oldest_tid}]=0 v17)=0 v17) v1 ==== 134+0+0 (secure 0 0 0) 0x7ff8f004f720 con 0x7ff8fc06cf10 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 msgr2=0x7ff8e8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 0x7ff8e8062a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0c6af0 tx=0x7ff8ec0091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 msgr2=0x7ff8fc152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c4e0 tx=0x7ff8f000c7f0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 shutdown_connections 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff8e8060550 0x7ff8e8062a00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc159030 0x7ff8fc15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8fc0ca600 0x7ff8fc15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 --2- 172.21.15.57:0/3193692446 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff8fc06cf10 0x7ff8fc152520 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 >> 172.21.15.57:0/3193692446 conn(0x7ff8fc068650 msgr2=0x7ff8fc068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:33.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 shutdown_connections 2023-10-07T23:23:33.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.272+0000 7ff902822700 1 -- 172.21.15.57:0/3193692446 wait complete. 2023-10-07T23:23:33.291 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-07T23:23:33.292 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:23:33.292 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:33.301 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43566]: fuse finished with error 0 and tester_r 0 2023-10-07T23:23:33.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:23:33.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.303 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T23:23:33.385 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-07T23:23:33.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 Processor -- start 2023-10-07T23:23:33.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 -- start start 2023-10-07T23:23:33.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340afe90 0x7efe340caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efe340cc740 0x7efe34067c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe340cb440 con 0x7efe3415cb20 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3415a440 con 0x7efe340cc740 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.496+0000 7efe39a3a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe340cdbb0 con 0x7efe340afe90 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7efe340cc740 0x7efe34067c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe33fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7efe3415cb20 0x7efe340cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:37364/0 (socket says 172.21.15.57:37364) 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe33fff700 1 -- 172.21.15.57:0/222551783 learned_addr learned my addr 172.21.15.57:0/222551783 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:23:33.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe38a38700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340afe90 0x7efe340caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 -- 172.21.15.57:0/222551783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340afe90 msgr2=0x7efe340caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340afe90 0x7efe340caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 -- 172.21.15.57:0/222551783 >> v1:172.21.15.57:6789/0 conn(0x7efe3415cb20 legacy=0x7efe340cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 -- 172.21.15.57:0/222551783 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3415cfb0 con 0x7efe340cc740 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe337fe700 1 -- 172.21.15.57:0/222551783 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69083935 0 0) 0x7efe340cb440 con 0x7efe3415cb20 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39239700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340cc740 0x7efe34067c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe240098e0 tx=0x7efe240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e531141a5d7d1c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe337fe700 1 -- 172.21.15.57:0/222551783 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24009e80 con 0x7efe340cc740 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe337fe700 1 -- 172.21.15.57:0/222551783 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7efe2400b860 con 0x7efe340cc740 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe337fe700 1 -- 172.21.15.57:0/222551783 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400bbe0 con 0x7efe340cc740 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 -- 172.21.15.57:0/222551783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340cc740 msgr2=0x7efe34067c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340cc740 0x7efe34067c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7efe240098e0 tx=0x7efe240091a0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 -- 172.21.15.57:0/222551783 shutdown_connections 2023-10-07T23:23:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340cc740 0x7efe34067c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 --2- 172.21.15.57:0/222551783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340afe90 0x7efe340caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 -- 172.21.15.57:0/222551783 >> 172.21.15.57:0/222551783 conn(0x7efe340692f0 msgr2=0x7efe340696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 -- 172.21.15.57:0/222551783 shutdown_connections 2023-10-07T23:23:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.500+0000 7efe39a3a700 1 -- 172.21.15.57:0/222551783 wait complete. 2023-10-07T23:23:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 Processor -- start 2023-10-07T23:23:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 -- start start 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340cc740 0x7efe340b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3415cb20 0x7efe340b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe340cb370 con 0x7efe340afe90 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe34147ba0 con 0x7efe3415cb20 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39a3a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3415ea10 con 0x7efe340cc740 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37380/0 (socket says 172.21.15.57:37380) 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe33fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340cc740 0x7efe340b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 -- 172.21.15.57:0/2502166742 learned_addr learned my addr 172.21.15.57:0/2502166742 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe39239700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3415cb20 0x7efe340b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 -- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340cc740 msgr2=0x7efe340b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340cc740 0x7efe340b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 -- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3415cb20 msgr2=0x7efe340b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3415cb20 0x7efe340b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 -- 172.21.15.57:0/2502166742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe24009420 con 0x7efe340afe90 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe38a38700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d860 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.504+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0125f0 con 0x7efe340afe90 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7efe2c012750 con 0x7efe340afe90 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3415ecf0 con 0x7efe340afe90 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c012b10 con 0x7efe340afe90 2023-10-07T23:23:33.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3415f2b0 con 0x7efe340afe90 2023-10-07T23:23:33.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe341586a0 con 0x7efe340afe90 2023-10-07T23:23:33.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.508+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7efe2c01e460 con 0x7efe340afe90 2023-10-07T23:23:33.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.512+0000 7efe30ff9700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 0x7efe18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:33.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.512+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe2c07cdb0 con 0x7efe340afe90 2023-10-07T23:23:33.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.512+0000 7efe33fff700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 0x7efe18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:33.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.512+0000 7efe33fff700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 0x7efe18062a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe1c002250 tx=0x7efe1c001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:33.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.516+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efe341586a0 con 0x7efe340afe90 2023-10-07T23:23:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.716+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efe341586a0 con 0x7efe340afe90 2023-10-07T23:23:33.719 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.716+0000 7efe30ff9700 1 -- 172.21.15.57:0/2502166742 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+73 (secure 0 0 0) 0x7efe341586a0 con 0x7efe340afe90 2023-10-07T23:23:33.720 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-07T23:23:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 msgr2=0x7efe18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 0x7efe18062a00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe1c002250 tx=0x7efe1c001a20 comp rx=0 tx=0).stop 2023-10-07T23:23:33.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 msgr2=0x7efe34147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00d860 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 shutdown_connections 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7efe18060550 0x7efe18062a00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe3415cb20 0x7efe340b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7efe340cc740 0x7efe340b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 --2- 172.21.15.57:0/2502166742 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efe340afe90 0x7efe34147660 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 >> 172.21.15.57:0/2502166742 conn(0x7efe340692f0 msgr2=0x7efe3415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 shutdown_connections 2023-10-07T23:23:33.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:33.720+0000 7efe39a3a700 1 -- 172.21.15.57:0/2502166742 wait complete. 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:33.734 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-07T23:23:33.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-07T23:23:33.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-07T23:23:33.735 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:33.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.735 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2023-10-07T23:23:33.743 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-07T23:23:33.743 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-07T23:23:33.743 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2023-10-07T23:23:33.743 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42422sec preferred_lft 42422sec 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2023-10-07T23:23:33.744 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 02:42:22:80:74:34 brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.745 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::88e7:63ff:fe49:2a21/64 scope link 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 5a:76:86:e5:a8:05 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::5876:86ff:fee5:a805/64 scope link 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0e:75:b1:f6:55:2a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::c75:b1ff:fef6:552a/64 scope link 2023-10-07T23:23:33.746 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2023-10-07T23:23:33.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.747 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-07T23:23:33.759 INFO:teuthology.orchestra.run.smithi057.stdout:1 2023-10-07T23:23:33.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.761 DEBUG:teuthology.orchestra.run.smithi057:> route 2023-10-07T23:23:33.813 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2023-10-07T23:23:33.813 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-07T23:23:33.813 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-07T23:23:33.814 INFO:teuthology.orchestra.run.smithi057.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-07T23:23:33.814 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-07T23:23:33.814 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-07T23:23:33.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.814 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec bash -c ' 2023-10-07T23:23:33.814 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2023-10-07T23:23:33.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-07T23:23:33.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-07T23:23:33.814 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-07T23:23:33.815 DEBUG:teuthology.orchestra.run.smithi057:> ') 2023-10-07T23:23:33.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.855 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:23:33.904 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T23:23:33.904 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:23:33.904 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:33.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:23:33.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2023-10-07T23:23:33.923 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-07T23:23:33.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:33.923 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:33.930 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-07T23:23:33.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-07T23:23:33.930 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:33.938 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:33.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:33.999 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:34.014 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:34.017 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:34.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:34.018 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:34.025 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-07T23:23:34.026 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-07T23:23:34.071 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:34.088 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:34.088 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:34.097 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:34.098 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:34.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:34.098 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:34.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:23:34.160+0000 7f4295296380 -1 init, newargv = 0x55efc8de6d70 newargc=15 2023-10-07T23:23:34.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43738]: starting ceph client 2023-10-07T23:23:34.172 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43738]: starting fuse 2023-10-07T23:23:35.207 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2023-10-07T23:23:35.223 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-07T23:23:35.223 DEBUG:teuthology.orchestra.run.smithi057:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-07T23:23:35.282 INFO:teuthology.orchestra.run.smithi057.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-07T23:23:35.283 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-07T23:23:35.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:35.284 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2023-10-07T23:23:35.290 INFO:teuthology.orchestra.run.smithi057.stdout:52 2023-10-07T23:23:35.291 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2023-10-07T23:23:35.291 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:23:35.291 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2023-10-07T23:23:35.291 DEBUG:teuthology.orchestra.run.smithi057:> import re 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2023-10-07T23:23:35.292 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> return f 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.0")) 2023-10-07T23:23:35.293 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:23:35.376 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.0.43738.asok 2023-10-07T23:23:35.561 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43738.asok 2023-10-07T23:23:35.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:35.562 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43738.asok status 2023-10-07T23:23:35.704 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:23:35.704 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2023-10-07T23:23:35.704 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "08cac0187919fc3a247efdf445f652e2fb30a647", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 17.2.6-1320-g08cac018 (08cac0187919fc3a247efdf445f652e2fb30a647) quincy (stable)", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "0", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "43738", 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 5565, 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2023-10-07T23:23:35.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 5565 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1259509611 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: } 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2023-10-07T23:23:35.706 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1259509611 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.5565 192.168.0.1:0/1259509611", 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/1259509611", 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 19, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 47, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2023-10-07T23:23:35.707 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:23:35.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:35.714 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2023-10-07T23:23:35.714 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:35.722 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2023-10-07T23:23:35.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:35.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-07T23:23:35.722 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-07T23:23:35.782 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_completed_requests 10000 2023-10-07T23:23:35.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 Processor -- start 2023-10-07T23:23:35.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- start start 2023-10-07T23:23:35.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4bc155610 0x7fc4bc1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc4bc1596f0 0x7fc4bc0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc159c60 con 0x7fc4bc0c9580 2023-10-07T23:23:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc159f40 con 0x7fc4bc155610 2023-10-07T23:23:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc15a220 con 0x7fc4bc1596f0 2023-10-07T23:23:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4bc155610 0x7fc4bc1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fc4bc155610 0x7fc4bc1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37384/0 (socket says 172.21.15.57:37384) 2023-10-07T23:23:35.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 -- 172.21.15.57:0/1128072891 learned_addr learned my addr 172.21.15.57:0/1128072891 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4bbfff700 1 --2- 172.21.15.57:0/1128072891 >> v2:172.21.15.154:3300/0 conn(0x7fc4bc1596f0 0x7fc4bc0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 -- 172.21.15.57:0/1128072891 >> v2:172.21.15.154:3300/0 conn(0x7fc4bc1596f0 msgr2=0x7fc4bc0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:23:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 --2- 172.21.15.57:0/1128072891 >> v2:172.21.15.154:3300/0 conn(0x7fc4bc1596f0 0x7fc4bc0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 -- 172.21.15.57:0/1128072891 >> v1:172.21.15.154:6789/0 conn(0x7fc4bc0c9580 legacy=0x7fc4bc156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:35.901 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 -- 172.21.15.57:0/1128072891 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc15a500 con 0x7fc4bc155610 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c0b69700 1 --2- 172.21.15.57:0/1128072891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc155610 0x7fc4bc1559e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc06c540 tx=0x7fc4ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6014257c1502f4a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4bb7fe700 1 -- 172.21.15.57:0/1128072891 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00b860 con 0x7fc4bc155610 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4bb7fe700 1 -- 172.21.15.57:0/1128072891 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc4ac00b9c0 con 0x7fc4bc155610 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4bb7fe700 1 -- 172.21.15.57:0/1128072891 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00bd60 con 0x7fc4bc155610 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/1128072891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc155610 msgr2=0x7fc4bc1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/1128072891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc155610 0x7fc4bc1559e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc06c540 tx=0x7fc4ac009250 comp rx=0 tx=0).stop 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/1128072891 shutdown_connections 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/1128072891 >> v2:172.21.15.154:3300/0 conn(0x7fc4bc1596f0 0x7fc4bc0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:35.902 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/1128072891 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc155610 0x7fc4bc1559e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/1128072891 >> 172.21.15.57:0/1128072891 conn(0x7fc4bc0670e0 msgr2=0x7fc4bc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/1128072891 shutdown_connections 2023-10-07T23:23:35.903 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.896+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/1128072891 wait complete. 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 Processor -- start 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- start start 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4bc0c9580 0x7fc4bc169550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 0x7fc4bc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.905 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc16cf80 con 0x7fc4bc1596f0 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc16d260 con 0x7fc4bc0c9580 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc16d540 con 0x7fc4bc155610 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37400/0 (socket says 172.21.15.57:37400) 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 -- 172.21.15.57:0/880375356 learned_addr learned my addr 172.21.15.57:0/880375356 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4bbfff700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 0x7fc4bc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c0b69700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4bc0c9580 0x7fc4bc169550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 -- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 msgr2=0x7fc4bc0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:35.906 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 0x7fc4bc0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 -- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4bc0c9580 msgr2=0x7fc4bc169550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4bc0c9580 0x7fc4bc169550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 -- 172.21.15.57:0/880375356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0094f0 con 0x7fc4bc1596f0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4bbfff700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 0x7fc4bc0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c136a700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c840 tx=0x7fc4b4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4012630 con 0x7fc4bc1596f0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fc4b40127b0 con 0x7fc4bc1596f0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4012b70 con 0x7fc4bc1596f0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc16d820 con 0x7fc4bc1596f0 2023-10-07T23:23:35.907 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.900+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc16cb10 con 0x7fc4bc1596f0 2023-10-07T23:23:35.908 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.904+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc16da40 con 0x7fc4bc1596f0 2023-10-07T23:23:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.904+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fc4b4003030 con 0x7fc4bc1596f0 2023-10-07T23:23:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.904+0000 7fc4b8ff9700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 0x7fc49c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:23:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.904+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4b407c990 con 0x7fc4bc1596f0 2023-10-07T23:23:35.909 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.904+0000 7fc4c0b69700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 0x7fc49c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:23:35.910 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.908+0000 7fc4c0b69700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 0x7fc49c062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0090b0 tx=0x7fc4ac009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:23:35.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:35.912+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc4b4050a90 con 0x7fc4bc1596f0 2023-10-07T23:23:36.080 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.076+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mds_max_completed_requests}] v 0) v1 -- 0x7fc4bc0711c0 con 0x7fc4bc1596f0 2023-10-07T23:23:36.086 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4b8ff9700 1 -- 172.21.15.57:0/880375356 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_completed_requests}]=0 v18)=0 v18) v1 ==== 128+0+0 (secure 0 0 0) 0x7fc4b4050620 con 0x7fc4bc1596f0 2023-10-07T23:23:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 msgr2=0x7fc49c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 0x7fc49c062a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0090b0 tx=0x7fc4ac009250 comp rx=0 tx=0).stop 2023-10-07T23:23:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 msgr2=0x7fc4bc16c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:23:36.089 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c840 tx=0x7fc4b4001670 comp rx=0 tx=0).stop 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 shutdown_connections 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc4bc1596f0 0x7fc4bc16c010 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc4bc155610 0x7fc4bc0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fc49c060550 0x7fc49c062a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 --2- 172.21.15.57:0/880375356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4bc0c9580 0x7fc4bc169550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 >> 172.21.15.57:0/880375356 conn(0x7fc4bc0670e0 msgr2=0x7fc4bc156620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 shutdown_connections 2023-10-07T23:23:36.090 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:23:36.084+0000 7fc4c1b6b700 1 -- 172.21.15.57:0/880375356 wait complete. 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> n = 10100 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:23:36.102 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file1" 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:23:36.103 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:23:36.104 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:23:36.105 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:23:36.105 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:23:36.105 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:23:48.939 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:23:48.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:24:19.237 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:24:19.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> n = 5 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:24:32.234 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:24:32.235 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:24:32.236 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:24:32.237 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:24:32.694 INFO:tasks.ceph_test_case:waiting 1000s for health warning matching MDS_CLIENT_OLDEST_TID 2023-10-07T23:24:32.695 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-07T23:24:32.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 Processor -- start 2023-10-07T23:24:32.810 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 -- start start 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 0x7f3dd006d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 0x7f3dd00ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd00abe60 con 0x7f3dd0150f30 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd006e1b0 con 0x7f3dd0151300 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd62fc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0068570 con 0x7f3dd006dd50 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd52fa700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f3dd0150f30 0x7f3dd00672f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:48600/0 (socket says 172.21.15.57:48600) 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd52fa700 1 -- 172.21.15.57:0/1442014338 learned_addr learned my addr 172.21.15.57:0/1442014338 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:24:32.813 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd4af9700 1 --2- 172.21.15.57:0/1442014338 >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 0x7f3dd006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dd5afb700 1 --2- 172.21.15.57:0/1442014338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 0x7f3dd00ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86052441 0 0) 0x7f3dd00abe60 con 0x7f3dd0150f30 2023-10-07T23:24:32.814 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.807+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dbc003570 con 0x7f3dd0150f30 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd5afb700 1 -- 172.21.15.57:0/1442014338 >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 msgr2=0x7f3dd006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506646197 0 0) 0x7f3dbc003570 con 0x7f3dd0150f30 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd5afb700 1 --2- 172.21.15.57:0/1442014338 >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 0x7f3dd006d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd5afb700 1 -- 172.21.15.57:0/1442014338 >> v1:172.21.15.57:6789/0 conn(0x7f3dd0150f30 legacy=0x7f3dd00672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd5afb700 1 -- 172.21.15.57:0/1442014338 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00c9f40 con 0x7f3dd006dd50 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd4af9700 1 --2- 172.21.15.57:0/1442014338 >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 0x7f3dd006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd5afb700 1 --2- 172.21.15.57:0/1442014338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 0x7f3dd00ca3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009e60 tx=0x7f3dc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106aaaaad42b28ec server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc800e070 con 0x7f3dd006dd50 2023-10-07T23:24:32.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f3dc800b830 con 0x7f3dd006dd50 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dc7fff700 1 -- 172.21.15.57:0/1442014338 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc001490 con 0x7f3dd006dd50 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- 172.21.15.57:0/1442014338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 msgr2=0x7f3dd00ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/1442014338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 0x7f3dd00ca3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009e60 tx=0x7f3dc8009780 comp rx=0 tx=0).stop 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- 172.21.15.57:0/1442014338 shutdown_connections 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/1442014338 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd006dd50 0x7f3dd00ca3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/1442014338 >> v2:172.21.15.154:3300/0 conn(0x7f3dd0151300 0x7f3dd006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- 172.21.15.57:0/1442014338 >> 172.21.15.57:0/1442014338 conn(0x7f3dd00c9750 msgr2=0x7f3dd006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- 172.21.15.57:0/1442014338 shutdown_connections 2023-10-07T23:24:32.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- 172.21.15.57:0/1442014338 wait complete. 2023-10-07T23:24:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 Processor -- start 2023-10-07T23:24:32.818 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.811+0000 7f3dd62fc700 1 -- start start 2023-10-07T23:24:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd0150f30 0x7f3dd00c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd0151300 0x7f3dd00c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00c3ad0 con 0x7f3dd006dd50 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd015f450 con 0x7f3dd0151300 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00af710 con 0x7f3dd0150f30 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42686/0 (socket says 172.21.15.57:42686) 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 -- 172.21.15.57:0/2068320748 learned_addr learned my addr 172.21.15.57:0/2068320748 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd5afb700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd0151300 0x7f3dd00c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd4af9700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd0150f30 0x7f3dd00c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 -- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd0150f30 msgr2=0x7f3dd00c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd0150f30 0x7f3dd00c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:32.821 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 -- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd0151300 msgr2=0x7f3dd00c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd0151300 0x7f3dd00c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 -- 172.21.15.57:0/2068320748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0009420 con 0x7f3dd006dd50 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd52fa700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00cb90 tx=0x7f3dcc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007150 con 0x7f3dd006dd50 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f3dcc0072d0 con 0x7f3dd006dd50 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc8009a20 con 0x7f3dd006dd50 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007690 con 0x7f3dd006dd50 2023-10-07T23:24:32.822 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.815+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd015ee90 con 0x7f3dd006dd50 2023-10-07T23:24:32.823 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.819+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3dcc004a40 con 0x7f3dd006dd50 2023-10-07T23:24:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.819+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd00711c0 con 0x7f3dd006dd50 2023-10-07T23:24:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.819+0000 7f3dc57fa700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 0x7f3db4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.819+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f3dcc07be50 con 0x7f3dd006dd50 2023-10-07T23:24:32.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.819+0000 7f3dd4af9700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 0x7f3db4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:32.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.823+0000 7f3dd4af9700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 0x7f3db4062a00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3dd006c590 tx=0x7f3dc00091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:32.834 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:32.831+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3dcc04c560 con 0x7f3dd006dd50 2023-10-07T23:24:33.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.055+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f3dd006b8c0 con 0x7f3dd006dd50 2023-10-07T23:24:33.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.055+0000 7f3dc57fa700 1 -- 172.21.15.57:0/2068320748 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+46 (secure 0 0 0) 0x7f3dcc018020 con 0x7f3dd006dd50 2023-10-07T23:24:33.061 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:24:33.061 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 msgr2=0x7f3db4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 0x7f3db4062a00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3dd006c590 tx=0x7f3dc00091a0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 msgr2=0x7f3dd015e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00cb90 tx=0x7f3dcc001ac0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 shutdown_connections 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3db4060550 0x7f3db4062a00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3dd0151300 0x7f3dd00c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3dd0150f30 0x7f3dd00c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 --2- 172.21.15.57:0/2068320748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3dd006dd50 0x7f3dd015e4f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 >> 172.21.15.57:0/2068320748 conn(0x7f3dd00c9750 msgr2=0x7f3dd00cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 shutdown_connections 2023-10-07T23:24:33.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:33.059+0000 7f3dd62fc700 1 -- 172.21.15.57:0/2068320748 wait complete. 2023-10-07T23:24:33.075 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2023-10-07T23:24:33.075 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=1000 retry_count=0)... 2023-10-07T23:24:38.076 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health detail --format=json 2023-10-07T23:24:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.191+0000 7f97f15bb700 1 Processor -- start 2023-10-07T23:24:38.196 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.191+0000 7f97f15bb700 1 -- start start 2023-10-07T23:24:38.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.191+0000 7f97f15bb700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f97ec150f30 0x7f97ec151300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f97ec0672f0 0x7f97ec06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0abe60 con 0x7f97ec067790 2023-10-07T23:24:38.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec068570 con 0x7f97ec0672f0 2023-10-07T23:24:38.197 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0c9f40 con 0x7f97ec150f30 2023-10-07T23:24:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f97ec0672f0 0x7f97ec06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97ebfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f97ec150f30 0x7f97ec151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f97ec0672f0 0x7f97ec06d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49664/0 (socket says 172.21.15.57:49664) 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 -- 172.21.15.57:0/1333326587 learned_addr learned my addr 172.21.15.57:0/1333326587 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f0dba700 1 --1- 172.21.15.57:0/1333326587 >> v1:172.21.15.154:6789/0 conn(0x7f97ec067790 0x7f97ec15a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:33276/0 (socket says 172.21.15.57:33276) 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 -- 172.21.15.57:0/1333326587 >> v2:172.21.15.154:3300/0 conn(0x7f97ec150f30 msgr2=0x7f97ec151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 --2- 172.21.15.57:0/1333326587 >> v2:172.21.15.154:3300/0 conn(0x7f97ec150f30 0x7f97ec151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 -- 172.21.15.57:0/1333326587 >> v1:172.21.15.154:6789/0 conn(0x7f97ec067790 legacy=0x7f97ec15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 -- 172.21.15.57:0/1333326587 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec067cd0 con 0x7f97ec0672f0 2023-10-07T23:24:38.199 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eb7fe700 1 --2- 172.21.15.57:0/1333326587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec06d810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0633a0 tx=0x7f97e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4670fe1830feda server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eaffd700 1 -- 172.21.15.57:0/1333326587 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0009e70 con 0x7f97ec0672f0 2023-10-07T23:24:38.200 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eaffd700 1 -- 172.21.15.57:0/1333326587 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f97e000b8f0 con 0x7f97ec0672f0 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97eaffd700 1 -- 172.21.15.57:0/1333326587 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000bc90 con 0x7f97ec0672f0 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- 172.21.15.57:0/1333326587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 msgr2=0x7f97ec06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 --2- 172.21.15.57:0/1333326587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec06d810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0633a0 tx=0x7f97e0009250 comp rx=0 tx=0).stop 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- 172.21.15.57:0/1333326587 shutdown_connections 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 --2- 172.21.15.57:0/1333326587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec06d810 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 --2- 172.21.15.57:0/1333326587 >> v2:172.21.15.154:3300/0 conn(0x7f97ec150f30 0x7f97ec151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- 172.21.15.57:0/1333326587 >> 172.21.15.57:0/1333326587 conn(0x7f97ec0c9750 msgr2=0x7f97ec06acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- 172.21.15.57:0/1333326587 shutdown_connections 2023-10-07T23:24:38.201 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.195+0000 7f97f15bb700 1 -- 172.21.15.57:0/1333326587 wait complete. 2023-10-07T23:24:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 Processor -- start 2023-10-07T23:24:38.204 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 -- start start 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec067790 0x7f97ec14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 0x7f97ec0ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0af8b0 con 0x7f97ec0672f0 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0afb90 con 0x7f97ec067790 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0afe70 con 0x7f97ec150f30 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49676/0 (socket says 172.21.15.57:49676) 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 -- 172.21.15.57:0/2254171627 learned_addr learned my addr 172.21.15.57:0/2254171627 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:24:38.207 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f0dba700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 0x7f97ec0ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97eb7fe700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec067790 0x7f97ec14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 -- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 msgr2=0x7f97ec0ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 0x7f97ec0ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 -- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec067790 msgr2=0x7f97ec14dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec067790 0x7f97ec14dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 -- 172.21.15.57:0/2254171627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00094f0 con 0x7f97ec0672f0 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f0dba700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 0x7f97ec0ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97ebfff700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00ec80 tx=0x7f97dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.208 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00c9d0 con 0x7f97ec0672f0 2023-10-07T23:24:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f97dc00cb30 con 0x7f97ec0672f0 2023-10-07T23:24:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec0b0150 con 0x7f97ec0672f0 2023-10-07T23:24:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.199+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc013470 con 0x7f97ec0672f0 2023-10-07T23:24:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec0af5a0 con 0x7f97ec0672f0 2023-10-07T23:24:38.209 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0c3f20 con 0x7f97ec0672f0 2023-10-07T23:24:38.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f97dc0137e0 con 0x7f97ec0672f0 2023-10-07T23:24:38.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97d3fff700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 0x7f97d8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f97dc07cc40 con 0x7f97ec0672f0 2023-10-07T23:24:38.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.203+0000 7f97eb7fe700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 0x7f97d8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.210 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.207+0000 7f97eb7fe700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 0x7f97d8062a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f97e00090b0 tx=0x7f97e0009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.217 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.211+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f97ec0c3f20 con 0x7f97ec0672f0 2023-10-07T23:24:38.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.435+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f97ec0c3f20 con 0x7f97ec0672f0 2023-10-07T23:24:38.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.435+0000 7f97d3fff700 1 -- 172.21.15.57:0/2254171627 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail", "format": "json"}]=0 v0) v1 ==== 92+0+322 (secure 0 0 0) 0x7f97dc017160 con 0x7f97ec0672f0 2023-10-07T23:24:38.440 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:24:38.441 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CLIENT_OLDEST_TID":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to advance oldest client/flush tid","count":1},"detail":[{"message":"mds.d(mds.0): Client smithi057:0 failing to advance its oldest client/flush tid. client_id: 5565"}],"muted":false}},"mutes":[]} 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 msgr2=0x7f97d8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 0x7f97d8062a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f97e00090b0 tx=0x7f97e0009250 comp rx=0 tx=0).stop 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 msgr2=0x7f97ec0af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00ec80 tx=0x7f97dc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 shutdown_connections 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f97ec150f30 0x7f97ec0ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.444 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f97d8060580 0x7f97d8062a30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ec067790 0x7f97ec14dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 --2- 172.21.15.57:0/2254171627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97ec0672f0 0x7f97ec0af060 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 >> 172.21.15.57:0/2254171627 conn(0x7f97ec0c9750 msgr2=0x7f97ec0ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 shutdown_connections 2023-10-07T23:24:38.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.439+0000 7f97f15bb700 1 -- 172.21.15.57:0/2254171627 wait complete. 2023-10-07T23:24:38.459 DEBUG:tasks.ceph_test_case:detail_strings: ['mds.d(mds.0): Client smithi057:0 failing to advance its oldest client/flush tid. client_id: 5565'] 2023-10-07T23:24:38.459 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-07T23:24:38.460 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_metadata_threshold 5000 2023-10-07T23:24:38.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.567+0000 7f9696fa4700 1 Processor -- start 2023-10-07T23:24:38.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.567+0000 7f9696fa4700 1 -- start start 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f96900c87d0 0x7f96900ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f96900c6d20 0x7f96900c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96900c6890 con 0x7f96900c8400 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900c5ff0 con 0x7f96900c87d0 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900ad610 con 0x7f96900c6d20 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f96900c87d0 0x7f96900ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9695fa2700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f96900c8400 0x7f96900acb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:49930/0 (socket says 172.21.15.57:49930) 2023-10-07T23:24:38.575 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9695fa2700 1 -- 172.21.15.57:0/1989898957 learned_addr learned my addr 172.21.15.57:0/1989898957 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96967a3700 1 --2- 172.21.15.57:0/1989898957 >> v2:172.21.15.154:3300/0 conn(0x7f96900c6d20 0x7f96900c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 -- 172.21.15.57:0/1989898957 >> v2:172.21.15.154:3300/0 conn(0x7f96900c6d20 msgr2=0x7f96900c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 --2- 172.21.15.57:0/1989898957 >> v2:172.21.15.154:3300/0 conn(0x7f96900c6d20 0x7f96900c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 -- 172.21.15.57:0/1989898957 >> v1:172.21.15.57:6789/0 conn(0x7f96900c8400 legacy=0x7f96900acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 -- 172.21.15.57:0/1989898957 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900c8c20 con 0x7f96900c87d0 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9694fa0700 1 -- 172.21.15.57:0/1989898957 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627129081 0 0) 0x7f96900c6890 con 0x7f96900c8400 2023-10-07T23:24:38.576 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f96957a1700 1 --2- 172.21.15.57:0/1989898957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900ae860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f969006d900 tx=0x7f96800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936c71c49b92d310 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9694fa0700 1 -- 172.21.15.57:0/1989898957 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680011630 con 0x7f96900c87d0 2023-10-07T23:24:38.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9694fa0700 1 -- 172.21.15.57:0/1989898957 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f9680011790 con 0x7f96900c87d0 2023-10-07T23:24:38.577 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9694fa0700 1 -- 172.21.15.57:0/1989898957 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680011b30 con 0x7f96900c87d0 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- 172.21.15.57:0/1989898957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 msgr2=0x7f96900ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 --2- 172.21.15.57:0/1989898957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900ae860 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f969006d900 tx=0x7f96800091a0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- 172.21.15.57:0/1989898957 shutdown_connections 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 --2- 172.21.15.57:0/1989898957 >> v2:172.21.15.154:3300/0 conn(0x7f96900c6d20 0x7f96900c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 --2- 172.21.15.57:0/1989898957 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900ae860 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- 172.21.15.57:0/1989898957 >> 172.21.15.57:0/1989898957 conn(0x7f96900670e0 msgr2=0x7f96900674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- 172.21.15.57:0/1989898957 shutdown_connections 2023-10-07T23:24:38.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.571+0000 7f9696fa4700 1 -- 172.21.15.57:0/1989898957 wait complete. 2023-10-07T23:24:38.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 Processor -- start 2023-10-07T23:24:38.579 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 -- start start 2023-10-07T23:24:38.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 0x7f96900c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96900c8400 0x7f9690169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969016a1f0 con 0x7f96900c87d0 2023-10-07T23:24:38.581 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969015ee50 con 0x7f96900c6d20 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9696fa4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900c26f0 con 0x7f96900c8400 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f96967a3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f96967a3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49700/0 (socket says 172.21.15.57:49700) 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f96967a3700 1 -- 172.21.15.57:0/4211368043 learned_addr learned my addr 172.21.15.57:0/4211368043 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f96957a1700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96900c8400 0x7f9690169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.582 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.575+0000 7f9695fa2700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 0x7f96900c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 -- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96900c8400 msgr2=0x7f9690169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96900c8400 0x7f9690169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 -- 172.21.15.57:0/4211368043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 msgr2=0x7f96900aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.583 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 -- 172.21.15.57:0/4211368043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680009420 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9695fa2700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 0x7f96900c2080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f968c0011e0 tx=0x7f968c007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c005b30 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f968c005c90 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969015e800 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c007300 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969015e4d0 con 0x7f96900c6d20 2023-10-07T23:24:38.584 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96900711c0 con 0x7f96900c6d20 2023-10-07T23:24:38.585 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.579+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f968c01b3b0 con 0x7f96900c6d20 2023-10-07T23:24:38.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.583+0000 7f9685ffb700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 0x7f9674062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:24:38.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.583+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f968c07c0d0 con 0x7f96900c6d20 2023-10-07T23:24:38.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.583+0000 7f96957a1700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 0x7f9674062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:24:38.586 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.583+0000 7f96957a1700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 0x7f9674062a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9680009270 tx=0x7f96800091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:24:38.596 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.591+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f968c04c6e0 con 0x7f96900c6d20 2023-10-07T23:24:38.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.755+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mds_session_metadata_threshold}] v 0) v1 -- 0x7f96900c6240 con 0x7f96900c6d20 2023-10-07T23:24:38.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.767+0000 7f9685ffb700 1 -- 172.21.15.57:0/4211368043 <== mon.1 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_metadata_threshold}]=0 v19)=0 v19) v1 ==== 131+0+0 (secure 0 0 0) 0x7f968c017020 con 0x7f96900c6d20 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 msgr2=0x7f9674062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 0x7f9674062a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9680009270 tx=0x7f96800091a0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 msgr2=0x7f96900c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 0x7f96900c2080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f968c0011e0 tx=0x7f968c007d30 comp rx=0 tx=0).stop 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 shutdown_connections 2023-10-07T23:24:38.775 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f96900c87d0 0x7f96900aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f96900c8400 0x7f9690169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f9674060550 0x7f9674062a00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 --2- 172.21.15.57:0/4211368043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96900c6d20 0x7f96900c2080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 >> 172.21.15.57:0/4211368043 conn(0x7f96900670e0 msgr2=0x7f969006adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 shutdown_connections 2023-10-07T23:24:38.776 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:24:38.771+0000 7f9696fa4700 1 -- 172.21.15.57:0/4211368043 wait complete. 2023-10-07T23:24:38.788 DEBUG:teuthology.orchestra.run.smithi057:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-07T23:24:38.788 DEBUG:teuthology.orchestra.run.smithi057:> import os 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> import uuid 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> n = 100000 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = False 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> if 0 > 0: 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> create_hard_links = True 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> path = "/home/ubuntu/cephtest/mnt.0/testdir/file2" 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> dpath = os.path.dirname(path) 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> fnameprefix = os.path.basename(path) 2023-10-07T23:24:38.789 DEBUG:teuthology.orchestra.run.smithi057:> os.makedirs(dpath, exist_ok=True) 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> try: 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> dirfd = os.open(dpath, os.O_DIRECTORY) 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> for i in range(n): 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> with open(fpath, '"'"'w'"'"') as f: 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> f.write(f"{i}") 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> if True: 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> f.flush() 2023-10-07T23:24:38.790 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(f.fileno()) 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> os.unlink(fpath) 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> if create_hard_links: 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> for j in range(0): 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> if False: 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> os.fsync(dirfd) 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> finally: 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> os.close(dirfd) 2023-10-07T23:24:38.791 DEBUG:teuthology.orchestra.run.smithi057:> ' 2023-10-07T23:24:49.314 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:24:49.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:25:19.522 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:25:19.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:25:49.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:25:49.624 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:26:19.720 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:26:19.723 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:26:49.792 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:26:49.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:27:19.845 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:27:19.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:27:49.912 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:27:49.914 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:28:19.958 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:28:19.962 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:28:50.003 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:28:50.006 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:29:20.392 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:29:20.395 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:29:50.439 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:29:50.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:30:19.014 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-07T23:30:19.011+0000 7fc367b62700 -1 mds.0.sessionmap save: session (client.5565) exceeds sesion metadata threshold - blocklisting 2023-10-07T23:30:19.017 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-07T23:30:19.015+0000 7fc367b62700 -1 mds.0.sessionmap save_if_dirty: session (client.5565) exceeds sesion metadata threshold - blocklisting 2023-10-07T23:30:19.030 INFO:tasks.ceph.mds.d.smithi154.stderr:2023-10-07T23:30:19.027+0000 7fc367b62700 -1 mds.0.sessionmap save_if_dirty: session (client.5565) exceeds sesion metadata threshold - blocklisting 2023-10-07T23:30:20.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:30:20.032+0000 7f427a7fc700 -1 client.5565 I was blocklisted at osd epoch 50 2023-10-07T23:30:20.490 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:30:20.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:30:20.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:2023-10-07T23:30:20.492+0000 7f427a7fc700 -1 client.5565 remove_session_caps still has dirty|flushing caps on 0x10000008962.head(faked_ino=0 nref=14 ll_ref=1 cap_refs={4=0,1024=0,4096=0,8192=0} open={2=1} mode=100664 size=5/4194304 nlink=1 btime=2023-10-07T23:30:20.019136+0000 mtime=2023-10-07T23:30:20.023247+0000 ctime=2023-10-07T23:30:20.023247+0000 change_attr=1 caps=- flushing_caps=Fw objectset[0x10000008962 ts 0/0 objects 1 dirty_or_tx 0] parents=0x10000000000.head["file2_24569"] 0x7f42776308e0) 2023-10-07T23:30:20.751 INFO:teuthology.orchestra.run.smithi057.stderr:Traceback (most recent call last): 2023-10-07T23:30:20.751 INFO:teuthology.orchestra.run.smithi057.stderr: File "", line 20, in 2023-10-07T23:30:20.751 INFO:teuthology.orchestra.run.smithi057.stderr:BrokenPipeError: [Errno 108] Cannot send after transport endpoint shutdown: '/home/ubuntu/cephtest/mnt.0/testdir/file2_24570' 2023-10-07T23:30:20.888 INFO:teuthology.orchestra.run.smithi057.stderr:daemon-helper: command failed with exit status 1 2023-10-07T23:30:20.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:30:20.898 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:20.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 Processor -- start 2023-10-07T23:30:20.998 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 -- start start 2023-10-07T23:30:20.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f6838150150 0x7f683806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68381594f0 con 0x7f683814fd80 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6838159790 con 0x7f68380c93c0 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f683cbf8700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6838159a70 con 0x7f6838150150 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f68377fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f683814fd80 0x7f68380abb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46426/0 (socket says 172.21.15.57:46426) 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f68377fe700 1 -- 172.21.15.57:0/2718408784 learned_addr learned my addr 172.21.15.57:0/2718408784 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.992+0000 7f6836ffd700 1 --2- 172.21.15.57:0/2718408784 >> v2:172.21.15.57:3300/0 conn(0x7f6838150150 0x7f683806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f68367fc700 1 -- 172.21.15.57:0/2718408784 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601763258 0 0) 0x7f68381594f0 con 0x7f683814fd80 2023-10-07T23:30:21.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f68367fc700 1 -- 172.21.15.57:0/2718408784 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6820003570 con 0x7f683814fd80 2023-10-07T23:30:21.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f6836ffd700 1 -- 172.21.15.57:0/2718408784 >> v1:172.21.15.154:6789/0 conn(0x7f68380c93c0 legacy=0x7f683806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f6836ffd700 1 -- 172.21.15.57:0/2718408784 >> v1:172.21.15.57:6789/0 conn(0x7f683814fd80 legacy=0x7f68380abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f6836ffd700 1 -- 172.21.15.57:0/2718408784 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838159d50 con 0x7f6838150150 2023-10-07T23:30:21.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f6836ffd700 1 --2- 172.21.15.57:0/2718408784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6838150150 0x7f683806c800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfca4451717045fa server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f68367fc700 1 -- 172.21.15.57:0/2718408784 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00b640 con 0x7f6838150150 2023-10-07T23:30:21.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f68367fc700 1 -- 172.21.15.57:0/2718408784 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f682c00b7a0 con 0x7f6838150150 2023-10-07T23:30:21.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f68367fc700 1 -- 172.21.15.57:0/2718408784 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00bb40 con 0x7f6838150150 2023-10-07T23:30:21.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 -- 172.21.15.57:0/2718408784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6838150150 msgr2=0x7f683806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 --2- 172.21.15.57:0/2718408784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6838150150 0x7f683806c800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 -- 172.21.15.57:0/2718408784 shutdown_connections 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 --2- 172.21.15.57:0/2718408784 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6838150150 0x7f683806c800 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 -- 172.21.15.57:0/2718408784 >> 172.21.15.57:0/2718408784 conn(0x7f6838067b80 msgr2=0x7f68380c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 -- 172.21.15.57:0/2718408784 shutdown_connections 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:20.996+0000 7f683cbf8700 1 -- 172.21.15.57:0/2718408784 wait complete. 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 Processor -- start 2023-10-07T23:30:21.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- start start 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f683814fd80 0x7f68380af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 0x7f6838149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380aaee0 con 0x7f68380c93c0 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380aa2d0 con 0x7f683814fd80 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380aa5b0 con 0x7f6838150150 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.007 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60836/0 (socket says 172.21.15.57:60836) 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 -- 172.21.15.57:0/3823530099 learned_addr learned my addr 172.21.15.57:0/3823530099 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f6836ffd700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f683814fd80 0x7f68380af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f6837fff700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 0x7f6838149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 -- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 msgr2=0x7f6838149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 0x7f6838149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 -- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f683814fd80 msgr2=0x7f68380af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f683814fd80 0x7f68380af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 -- 172.21.15.57:0/3823530099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c009420 con 0x7f68380c93c0 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f6837fff700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 0x7f6838149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f68377fe700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f682800c7c0 tx=0x7f6828007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828005b30 con 0x7f68380c93c0 2023-10-07T23:30:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f6828005c90 con 0x7f68380c93c0 2023-10-07T23:30:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380aa890 con 0x7f68380c93c0 2023-10-07T23:30:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828007340 con 0x7f68380c93c0 2023-10-07T23:30:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.000+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380afec0 con 0x7f68380c93c0 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f682801b400 con 0x7f68380c93c0 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68380711c0 con 0x7f68380c93c0 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f681f7fe700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 0x7f6824062b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f682807c440 con 0x7f68380c93c0 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f6836ffd700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 0x7f6824062b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.004+0000 7f6836ffd700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 0x7f6824062b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f682c0093f0 tx=0x7f682c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.016+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f682804ca30 con 0x7f68380c93c0 2023-10-07T23:30:21.218 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.212+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68380630a0 con 0x7f68380c93c0 2023-10-07T23:30:21.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f681f7fe700 1 -- 172.21.15.57:0/3823530099 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15506 (secure 0 0 0) 0x7f6828050050 con 0x7f68380c93c0 2023-10-07T23:30:21.220 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:21.220 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":50,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:20.008535+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:21.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 msgr2=0x7f6824062b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 0x7f6824062b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f682c0093f0 tx=0x7f682c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.223 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 msgr2=0x7f68380ab370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f682800c7c0 tx=0x7f6828007e00 comp rx=0 tx=0).stop 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 shutdown_connections 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6838150150 0x7f6838149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f68240606a0 0x7f6824062b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f683814fd80 0x7f68380af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 --2- 172.21.15.57:0/3823530099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68380c93c0 0x7f68380ab370 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 >> 172.21.15.57:0/3823530099 conn(0x7f6838067b80 msgr2=0x7f68380cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 shutdown_connections 2023-10-07T23:30:21.224 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.216+0000 7f683cbf8700 1 -- 172.21.15.57:0/3823530099 wait complete. 2023-10-07T23:30:21.237 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:21.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 Processor -- start 2023-10-07T23:30:21.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 -- start start 2023-10-07T23:30:21.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d700c9ad0 0x7f8d70157380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7006d990 con 0x7f8d701592b0 2023-10-07T23:30:21.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d70157140 con 0x7f8d700c9ad0 2023-10-07T23:30:21.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d74816700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7006cd80 con 0x7f8d70158ee0 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d700c9ad0 0x7f8d70157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:42734/0 (socket says 172.21.15.57:42734) 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d700c9ad0 0x7f8d70157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:54244/0 (socket says 172.21.15.57:54244) 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6f7fe700 1 -- 172.21.15.57:0/2160240622 learned_addr learned my addr 172.21.15.57:0/2160240622 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:21.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6effd700 1 --1- 172.21.15.57:0/2160240622 >> v1:172.21.15.154:6789/0 conn(0x7f8d701592b0 0x7f8d700c93c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.57:51716/0 (socket says 172.21.15.57:51716) 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 -- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 msgr2=0x7f8d70156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 -- 172.21.15.57:0/2160240622 >> v1:172.21.15.154:6789/0 conn(0x7f8d701592b0 legacy=0x7f8d700c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 -- 172.21.15.57:0/2160240622 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7006d060 con 0x7f8d700c9ad0 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6f7fe700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:21.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6ffff700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d700c9ad0 0x7f8d70157380 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e60 tx=0x7f8d68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f83778a222e3894 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6e7fc700 1 -- 172.21.15.57:0/2160240622 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800e070 con 0x7f8d700c9ad0 2023-10-07T23:30:21.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.320+0000 7f8d6e7fc700 1 -- 172.21.15.57:0/2160240622 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8d6800bb60 con 0x7f8d700c9ad0 2023-10-07T23:30:21.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d6e7fc700 1 -- 172.21.15.57:0/2160240622 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68012430 con 0x7f8d700c9ad0 2023-10-07T23:30:21.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d700c9ad0 msgr2=0x7f8d70157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d700c9ad0 0x7f8d70157380 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009e60 tx=0x7f8d68009780 comp rx=0 tx=0).stop 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- 172.21.15.57:0/2160240622 shutdown_connections 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d700c9ad0 0x7f8d70157380 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- 172.21.15.57:0/2160240622 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d70158ee0 0x7f8d70156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- 172.21.15.57:0/2160240622 >> 172.21.15.57:0/2160240622 conn(0x7f8d70067b80 msgr2=0x7f8d700c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- 172.21.15.57:0/2160240622 shutdown_connections 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- 172.21.15.57:0/2160240622 wait complete. 2023-10-07T23:30:21.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 Processor -- start 2023-10-07T23:30:21.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- start start 2023-10-07T23:30:21.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d70158ee0 0x7f8d70150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d701592b0 0x7f8d7014e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c6420 con 0x7f8d700c9ad0 2023-10-07T23:30:21.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70150d70 con 0x7f8d70158ee0 2023-10-07T23:30:21.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d74816700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70151750 con 0x7f8d701592b0 2023-10-07T23:30:21.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.324+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60848/0 (socket says 172.21.15.57:60848) 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 -- 172.21.15.57:0/3434074279 learned_addr learned my addr 172.21.15.57:0/3434074279 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6ffff700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d701592b0 0x7f8d7014e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6effd700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d70158ee0 0x7f8d70150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 -- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d701592b0 msgr2=0x7f8d7014e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d701592b0 0x7f8d7014e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 -- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d70158ee0 msgr2=0x7f8d70150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d70158ee0 0x7f8d70150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 -- 172.21.15.57:0/3434074279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6000b790 con 0x7f8d700c9ad0 2023-10-07T23:30:21.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d6f7fe700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000ed90 tx=0x7f8d6000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000ee90 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f8d6000dd80 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d68009a20 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60010650 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700c6140 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70151a30 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8d60010a40 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 0x7f8d5c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.328+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f8d6007cf00 con 0x7f8d700c9ad0 2023-10-07T23:30:21.336 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.332+0000 7f8d6effd700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 0x7f8d5c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:21.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.332+0000 7f8d6effd700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 0x7f8d5c062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d700c5aa0 tx=0x7f8d64000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:21.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.336+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8d6004d490 con 0x7f8d700c9ad0 2023-10-07T23:30:21.532 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.528+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d700711c0 con 0x7f8d700c9ad0 2023-10-07T23:30:21.534 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.528+0000 7f8d577fe700 1 -- 172.21.15.57:0/3434074279 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f8d60050ab0 con 0x7f8d700c9ad0 2023-10-07T23:30:21.534 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:21.534 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:24:35.918170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:30:21.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 msgr2=0x7f8d5c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.536 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 0x7f8d5c062a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d700c5aa0 tx=0x7f8d64000bc0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 msgr2=0x7f8d7014d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000ed90 tx=0x7f8d6000b4f0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 shutdown_connections 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8d701592b0 0x7f8d7014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8d5c060550 0x7f8d5c062a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d70158ee0 0x7f8d70150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 --2- 172.21.15.57:0/3434074279 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d700c9ad0 0x7f8d7014d080 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 >> 172.21.15.57:0/3434074279 conn(0x7f8d70067b80 msgr2=0x7f8d70153dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 shutdown_connections 2023-10-07T23:30:21.537 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:21.532+0000 7f8d74816700 1 -- 172.21.15.57:0/3434074279 wait complete. 2023-10-07T23:30:21.538 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2023-10-07T23:30:21.549 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2023-10-07T23:30:21.667 INFO:teuthology.orchestra.run.smithi154.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":8428,"msgr_send_messages":8422,"msgr_recv_bytes":2230315,"msgr_send_bytes":46042769,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":1.373749183,"msgr_running_send_time":0.598276838,"msgr_running_recv_time":0.257860540,"msgr_running_fast_dispatch_time":0.424659661,"msgr_send_messages_queue_lat":{"avgcount":8421,"sum":1.923144093,"avgtime":0.000228374},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"AsyncMessenger::Worker-1":{"msgr_recv_messages":160955,"msgr_send_messages":181185,"msgr_recv_bytes":47625981,"msgr_send_bytes":111720958,"msgr_created_connections":9,"msgr_active_connections":4,"msgr_running_total_time":19.053775414,"msgr_running_send_time":8.093882517,"msgr_running_recv_time":14.239410114,"msgr_running_fast_dispatch_time":0.383601769,"msgr_send_messages_queue_lat":{"avgcount":181184,"sum":8.111452228,"avgtime":0.000044769},"msgr_handle_ack_lat":{"avgcount":324820,"sum":0.100897805,"avgtime":0.000000310}},"AsyncMessenger::Worker-2":{"msgr_recv_messages":13907,"msgr_send_messages":13497,"msgr_recv_bytes":3562596,"msgr_send_bytes":89805555,"msgr_created_connections":6,"msgr_active_connections":4,"msgr_running_total_time":2.323348498,"msgr_running_send_time":0.952092030,"msgr_running_recv_time":0.418688245,"msgr_running_fast_dispatch_time":0.763319647,"msgr_send_messages_queue_lat":{"avgcount":13496,"sum":2.568366544,"avgtime":0.000190305},"msgr_handle_ack_lat":{"avgcount":460,"sum":0.000107894,"avgtime":0.000000234}},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49661,"sum":42.751092623,"avgtime":0.000860858}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000012739,"avgtime":0.000006369}},"mds":{"request":71255,"reply":71255,"reply_latency":{"avgcount":71255,"sum":97.048657966,"avgtime":0.001361990},"slow_reply":0,"forward":0,"dir_fetch":0,"dir_commit":21,"dir_split":1,"dir_merge":0,"inodes":34681,"inodes_top":0,"inodes_bottom":34681,"inodes_pin_tail":0,"inodes_pinned":34681,"inodes_expired":47,"inodes_with_caps":34672,"caps":34673,"subtrees":2,"traverse":107835,"traverse_hit":34689,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":297592,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":34640,"root_rbytes":150980,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":7,"ceph_cap_op_grant":34679,"ceph_cap_op_trunc":5,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":58915,"handle_client_caps_dirty":34675,"handle_client_cap_release":0,"process_request_cap_release":34672},"mds_cache":{"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":342,"recovery_completed":342,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":1,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":94152,"evex":2048,"evtrm":0,"ev":94152,"evexg":0,"evexd":2048,"segadd":130,"segex":2,"segtrm":0,"seg":130,"segexg":0,"segexd":2,"expos":4194304,"wrpos":186149755,"rdpos":0,"jlat":{"avgcount":24765,"sum":95.653509951,"avgtime":0.003862447},"replayed":0},"mds_mem":{"ino":34684,"ino+":34684,"ino-":0,"dir":20,"dir+":21,"dir-":1,"dn":34681,"dn+":34728,"dn-":47,"cap":1,"cap+":34674,"cap-":34673,"rss":208092,"heap":207072},"mds_server":{"dispatch_client_request":71263,"dispatch_server_request":0,"handle_client_request":71255,"handle_client_session":24620,"handle_peer_request":0,"req_create_latency":{"avgcount":34670,"sum":74.389368858,"avgtime":0.002145640},"req_getattr_latency":{"avgcount":4,"sum":0.001790084,"avgtime":0.000447521},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":36574,"sum":22.583265164,"avgtime":0.000617467},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":1,"sum":0.007428340,"avgtime":0.007428340},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":5,"sum":0.066238969,"avgtime":0.013247793},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000566551,"avgtime":0.000566551},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":16794,"average_load":8397,"avg_session_uptime":406,"mdthresh_evicted":3},"mempool":{"bloom_filter_bytes":41037,"bloom_filter_items":41037,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_Buffer_bytes":0,"bluestore_Buffer_items":0,"bluestore_Extent_bytes":0,"bluestore_Extent_items":0,"bluestore_Blob_bytes":0,"bluestore_Blob_items":0,"bluestore_SharedBlob_bytes":0,"bluestore_SharedBlob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":223009,"buffer_anon_items":85,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5432,"osdmap_items":79,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":106989080,"mds_co_items":693022,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":27502,"op_send_bytes":194607160,"op_resend":37,"op_reply":27502,"oplen_avg":{"avgcount":27502,"sum":31076},"op":27502,"op_r":342,"op_w":27160,"op_rmw":0,"op_pg":0,"osdop_stat":342,"osdop_create":1700,"osdop_read":0,"osdop_write":24765,"osdop_writefull":35,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":1,"osdop_delete":549,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":3400,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":5,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":50,"map_full":0,"map_inc":4,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":197,"omap_rd":0,"omap_del":82},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":183286,"get_sum":36912889,"get_or_fail_fail":0,"get_or_fail_success":183286,"take":0,"take_sum":0,"put":183286,"put_sum":36912889,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":27502,"take_sum":194607160,"put":27502,"put_sum":194607160,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":27502,"take_sum":27502,"put":27502,"put_sum":27502,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x562786a59e00":{"val":745797,"max":3758096384,"get_started":0,"get":94152,"get_sum":181955451,"get_or_fail_fail":0,"get_or_fail_success":94152,"take":0,"take_sum":0,"put":24765,"put_sum":181209654,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2023-10-07T23:30:21.669 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2230315, "msgr_recv_messages": 8428, "msgr_running_fast_dispatch_time": 0.424659661, "msgr_running_recv_time": 0.25786054, "msgr_running_send_time": 0.598276838, "msgr_running_total_time": 1.373749183, "msgr_send_bytes": 46042769, "msgr_send_messages": 8422, "msgr_send_messages_queue_lat": { "avgcount": 8421, "avgtime": 0.000228374, "sum": 1.923144093 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 324820, "avgtime": 3.1e-07, "sum": 0.100897805 }, "msgr_recv_bytes": 47625981, "msgr_recv_messages": 160955, "msgr_running_fast_dispatch_time": 0.383601769, "msgr_running_recv_time": 14.239410114, "msgr_running_send_time": 8.093882517, "msgr_running_total_time": 19.053775414, "msgr_send_bytes": 111720958, "msgr_send_messages": 181185, "msgr_send_messages_queue_lat": { "avgcount": 181184, "avgtime": 4.4769e-05, "sum": 8.111452228 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 460, "avgtime": 2.34e-07, "sum": 0.000107894 }, "msgr_recv_bytes": 3562596, "msgr_recv_messages": 13907, "msgr_running_fast_dispatch_time": 0.763319647, "msgr_running_recv_time": 0.418688245, "msgr_running_send_time": 0.95209203, "msgr_running_total_time": 2.323348498, "msgr_send_bytes": 89805555, "msgr_send_messages": 13497, "msgr_send_messages_queue_lat": { "avgcount": 13496, "avgtime": 0.000190305, "sum": 2.568366544 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49661, "avgtime": 0.000860858, "sum": 42.751092623 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 6.369e-06, "sum": 1.2739e-05 }, "queue_len": 0 }, "mds": { "caps": 34673, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 34679, "ceph_cap_op_revoke": 7, "ceph_cap_op_trunc": 5, "dir_commit": 21, "dir_fetch": 0, "dir_merge": 0, "dir_split": 1, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 58915, "handle_client_caps_dirty": 34675, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 34681, "inodes_bottom": 34681, "inodes_expired": 47, "inodes_pin_tail": 0, "inodes_pinned": 34681, "inodes_top": 0, "inodes_with_caps": 34672, "load_cent": 297592, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 34672, "q": 0, "reply": 71255, "reply_latency": { "avgcount": 71255, "avgtime": 0.00136199, "sum": 97.048657966 }, "request": 71255, "root_rbytes": 150980, "root_rfiles": 34640, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 107835, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 34689, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 1, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 342, "recovery_started": 342, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 94152, "evadd": 94152, "evex": 2048, "evexd": 2048, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 24765, "avgtime": 0.003862447, "sum": 95.653509951 }, "rdpos": 0, "replayed": 0, "seg": 130, "segadd": 130, "segex": 2, "segexd": 2, "segexg": 0, "segtrm": 0, "wrpos": 186149755 }, "mds_mem": { "cap": 1, "cap+": 34674, "cap-": 34673, "dir": 20, "dir+": 21, "dir-": 1, "dn": 34681, "dn+": 34728, "dn-": 47, "heap": 207072, "ino": 34684, "ino+": 34684, "ino-": 0, "rss": 208092 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 71263, "dispatch_server_request": 0, "handle_client_request": 71255, "handle_client_session": 24620, "handle_peer_request": 0, "req_create_latency": { "avgcount": 34670, "avgtime": 0.00214564, "sum": 74.389368858 }, "req_getattr_latency": { "avgcount": 4, "avgtime": 0.000447521, "sum": 0.001790084 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 36574, "avgtime": 0.000617467, "sum": 22.583265164 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 1, "avgtime": 0.00742834, "sum": 0.00742834 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 5, "avgtime": 0.013247793, "sum": 0.066238969 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000566551, "sum": 0.000566551 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 8397, "avg_session_uptime": 406, "mdthresh_evicted": 3, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 16794 }, "mempool": { "bloom_filter_bytes": 41037, "bloom_filter_items": 41037, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_Blob_bytes": 0, "bluestore_Blob_items": 0, "bluestore_Buffer_bytes": 0, "bluestore_Buffer_items": 0, "bluestore_Extent_bytes": 0, "bluestore_Extent_items": 0, "bluestore_SharedBlob_bytes": 0, "bluestore_SharedBlob_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 223009, "buffer_anon_items": 85, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 106989080, "mds_co_items": 693022, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5432, "osdmap_items": 79, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 50, "map_full": 0, "map_inc": 4, "omap_del": 82, "omap_rd": 0, "omap_wr": 197, "op": 27502, "op_active": 0, "op_laggy": 0, "op_pg": 0, "op_r": 342, "op_reply": 27502, "op_resend": 37, "op_rmw": 0, "op_send": 27502, "op_send_bytes": 194607160, "op_w": 27160, "oplen_avg": { "avgcount": 27502, "sum": 31076 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 1700, "osdop_delete": 549, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 5, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 3400, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 342, "osdop_truncate": 1, "osdop_watch": 0, "osdop_write": 24765, "osdop_writefull": 35, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 183286, "get_or_fail_fail": 0, "get_or_fail_success": 183286, "get_started": 0, "get_sum": 36912889, "max": 104857600, "put": 183286, "put_sum": 36912889, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 27502, "put_sum": 194607160, "take": 27502, "take_sum": 194607160, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 27502, "put_sum": 27502, "take": 27502, "take_sum": 27502, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x562786a59e00": { "get": 94152, "get_or_fail_fail": 0, "get_or_fail_success": 94152, "get_started": 0, "get_sum": 181955451, "max": 3758096384, "put": 24765, "put_sum": 181209654, "take": 0, "take_sum": 0, "val": 745797, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2023-10-07T23:30:21.671 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-07T23:30:21.671 DEBUG:teuthology.orchestra.run.smithi057:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:30:21.694 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-07T23:30:21.714 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-07T23:30:21.714 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2023-10-07T23:30:21.714 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:30:21.715 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-07T23:30:21.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi057.stderr:ceph-fuse[43738]: fuse finished with error 0 and tester_r 0 2023-10-07T23:30:21.729 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:30:21.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:21.730 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-07T23:30:21.862 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-10-07T23:30:21.863 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-07T23:30:21.863 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-07T23:30:21.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi154.stderr:ceph-fuse[39796]: fuse finished with error 0 and tester_r 0 2023-10-07T23:30:21.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:30:21.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:21.880 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-07T23:30:22.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:30:22.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:22.003 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-07T23:30:22.009 INFO:teuthology.orchestra.run.smithi057.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-07T23:30:22.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:30:22.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:30:22.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:22.011 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-07T23:30:22.017 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-07T23:30:22.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:30:22.018 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 Processor -- start 2023-10-07T23:30:22.133 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 -- start start 2023-10-07T23:30:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14c006bc90 0x7f14c00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.134 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c00c83b0 con 0x7f14c0158d60 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c00c8690 con 0x7f14c006bc90 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c6cba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00c8970 con 0x7f14c0159130 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14c006bc90 0x7f14c00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c64b9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14c006bc90 0x7f14c00c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:54264/0 (socket says 172.21.15.57:54264) 2023-10-07T23:30:22.135 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 -- 172.21.15.57:0/3841815357 learned_addr learned my addr 172.21.15.57:0/3841815357 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 -- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 msgr2=0x7f14c00c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 --2- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 -- 172.21.15.57:0/3841815357 >> v1:172.21.15.154:6789/0 conn(0x7f14c0158d60 legacy=0x7f14c006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 -- 172.21.15.57:0/3841815357 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c00c8c50 con 0x7f14c006bc90 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.128+0000 7f14c54b7700 1 --2- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c00c7580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0099b0 tx=0x7f14bc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e19a76ed44fb61 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.136 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c4cb6700 1 -- 172.21.15.57:0/3841815357 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc009ea0 con 0x7f14c006bc90 2023-10-07T23:30:22.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c4cb6700 1 -- 172.21.15.57:0/3841815357 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f14bc00b860 con 0x7f14c006bc90 2023-10-07T23:30:22.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c4cb6700 1 -- 172.21.15.57:0/3841815357 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00bc00 con 0x7f14c006bc90 2023-10-07T23:30:22.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 msgr2=0x7f14c00c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c00c7580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0099b0 tx=0x7f14bc009250 comp rx=0 tx=0).stop 2023-10-07T23:30:22.137 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- 172.21.15.57:0/3841815357 shutdown_connections 2023-10-07T23:30:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- 172.21.15.57:0/3841815357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c00c7580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- 172.21.15.57:0/3841815357 >> 172.21.15.57:0/3841815357 conn(0x7f14c00670e0 msgr2=0x7f14c00674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- 172.21.15.57:0/3841815357 shutdown_connections 2023-10-07T23:30:22.138 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- 172.21.15.57:0/3841815357 wait complete. 2023-10-07T23:30:22.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 Processor -- start 2023-10-07T23:30:22.139 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- start start 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c01675d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00c9e90 con 0x7f14c0158d60 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00cc7a0 con 0x7f14c006bc90 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.132+0000 7f14c6cba700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00c9560 con 0x7f14c0159130 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60860/0 (socket says 172.21.15.57:60860) 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 -- 172.21.15.57:0/752852688 learned_addr learned my addr 172.21.15.57:0/752852688 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:22.140 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c64b9700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.141 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c5cb8700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c01675d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 -- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 msgr2=0x7f14c00ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 -- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 msgr2=0x7f14c01675d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c01675d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 -- 172.21.15.57:0/752852688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0094f0 con 0x7f14c0158d60 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c54b7700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0090b0 tx=0x7f14bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc000d10 con 0x7f14c0158d60 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f14bc01f070 con 0x7f14c0158d60 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c00c9840 con 0x7f14c0158d60 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0266b0 con 0x7f14c0158d60 2023-10-07T23:30:22.142 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c0167b10 con 0x7f14c0158d60 2023-10-07T23:30:22.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c00c9cc0 con 0x7f14c0158d60 2023-10-07T23:30:22.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.136+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f14bc01a430 con 0x7f14c0158d60 2023-10-07T23:30:22.143 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.140+0000 7f14b5ffb700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 0x7f14a4062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.140+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f14bc021070 con 0x7f14c0158d60 2023-10-07T23:30:22.144 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.140+0000 7f14c5cb8700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 0x7f14a4062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.145 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.140+0000 7f14c5cb8700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 0x7f14a4062a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14c01595a0 tx=0x7f14b0000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.152 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.148+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f14bc04eac0 con 0x7f14c0158d60 2023-10-07T23:30:22.352 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.348+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14c00711c0 con 0x7f14c0158d60 2023-10-07T23:30:22.353 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.348+0000 7f14b5ffb700 1 -- 172.21.15.57:0/752852688 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f14bc015090 con 0x7f14c0158d60 2023-10-07T23:30:22.353 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:22.354 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:24:35.918170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:30:22.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.348+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 msgr2=0x7f14a4062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.348+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 0x7f14a4062a00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f14c01595a0 tx=0x7f14b0000bc0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.348+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 msgr2=0x7f14c00cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0090b0 tx=0x7f14bc009250 comp rx=0 tx=0).stop 2023-10-07T23:30:22.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 shutdown_connections 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f14c0159130 0x7f14c00ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f14c0158d60 0x7f14c00cbdc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f14a4060550 0x7f14a4062a00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 --2- 172.21.15.57:0/752852688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c006bc90 0x7f14c01675d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 >> 172.21.15.57:0/752852688 conn(0x7f14c00670e0 msgr2=0x7f14c006ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 shutdown_connections 2023-10-07T23:30:22.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.352+0000 7f14c6cba700 1 -- 172.21.15.57:0/752852688 wait complete. 2023-10-07T23:30:22.358 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2023-10-07T23:30:22.368 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:22.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.464+0000 7f505a7b3700 1 Processor -- start 2023-10-07T23:30:22.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.464+0000 7f505a7b3700 1 -- start start 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50540c6870 0x7f5054157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5054158380 con 0x7f50540abb30 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540c6d10 con 0x7f50540cba30 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540ae070 con 0x7f50540c6870 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f5058d50700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f50540abb30 0x7f50540abf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46438/0 (socket says 172.21.15.57:46438) 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f5058d50700 1 -- 172.21.15.57:0/684964124 learned_addr learned my addr 172.21.15.57:0/684964124 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:22.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f504bfff700 1 --2- 172.21.15.57:0/684964124 >> v2:172.21.15.154:3300/0 conn(0x7f50540c6870 0x7f5054157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f50537fe700 1 -- 172.21.15.57:0/684964124 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234426001 0 0) 0x7f504c001490 con 0x7f50540abb30 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f50537fe700 1 -- 172.21.15.57:0/684964124 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5040003570 con 0x7f50540abb30 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f504bfff700 1 -- 172.21.15.57:0/684964124 >> v1:172.21.15.154:6789/0 conn(0x7f50540cba30 legacy=0x7f50540cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f504bfff700 1 -- 172.21.15.57:0/684964124 >> v1:172.21.15.57:6789/0 conn(0x7f50540abb30 legacy=0x7f50540abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f504bfff700 1 -- 172.21.15.57:0/684964124 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505415ada0 con 0x7f50540c6870 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f504bfff700 1 --2- 172.21.15.57:0/684964124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540c6870 0x7f5054157e40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f503c0098e0 tx=0x7f503c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95cfbd6c1bfd6a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f50537fe700 1 -- 172.21.15.57:0/684964124 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00b640 con 0x7f50540c6870 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f50537fe700 1 -- 172.21.15.57:0/684964124 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f503c00b7a0 con 0x7f50540c6870 2023-10-07T23:30:22.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f50537fe700 1 -- 172.21.15.57:0/684964124 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00bb40 con 0x7f50540c6870 2023-10-07T23:30:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- 172.21.15.57:0/684964124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540c6870 msgr2=0x7f5054157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 --2- 172.21.15.57:0/684964124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540c6870 0x7f5054157e40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f503c0098e0 tx=0x7f503c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- 172.21.15.57:0/684964124 shutdown_connections 2023-10-07T23:30:22.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 --2- 172.21.15.57:0/684964124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540c6870 0x7f5054157e40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- 172.21.15.57:0/684964124 >> 172.21.15.57:0/684964124 conn(0x7f50540670e0 msgr2=0x7f50540674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- 172.21.15.57:0/684964124 shutdown_connections 2023-10-07T23:30:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.468+0000 7f505a7b3700 1 -- 172.21.15.57:0/684964124 wait complete. 2023-10-07T23:30:22.476 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 Processor -- start 2023-10-07T23:30:22.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 -- start start 2023-10-07T23:30:22.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540abb30 0x7f5054168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 0x7f50541490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 0x7f505414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541495e0 con 0x7f50540cba30 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541586c0 con 0x7f50540abb30 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f505a7b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50541589a0 con 0x7f50540c6870 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 0x7f505414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f504bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 0x7f50541490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f504bfff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 0x7f50541490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:42762/0 (socket says 172.21.15.57:42762) 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f504bfff700 1 -- 172.21.15.57:0/109533414 learned_addr learned my addr 172.21.15.57:0/109533414 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:22.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5053fff700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540abb30 0x7f5054168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 -- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 msgr2=0x7f50541490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 0x7f50541490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 -- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540abb30 msgr2=0x7f5054168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540abb30 0x7f5054168cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.472+0000 7f5058d50700 1 -- 172.21.15.57:0/109533414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c009420 con 0x7f50540cba30 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f5058d50700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 0x7f505414dd60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f504c00cb10 tx=0x7f504c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00a7e0 con 0x7f50540cba30 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f504c00a940 con 0x7f50540cba30 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c00acc0 con 0x7f50540cba30 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5054158c80 con 0x7f50540cba30 2023-10-07T23:30:22.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505414e2a0 con 0x7f50540cba30 2023-10-07T23:30:22.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5054159100 con 0x7f50540cba30 2023-10-07T23:30:22.483 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.476+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f504c007300 con 0x7f50540cba30 2023-10-07T23:30:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.480+0000 7f5050ff9700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 0x7f5034062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.480+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f504c07c830 con 0x7f50540cba30 2023-10-07T23:30:22.484 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.480+0000 7f5053fff700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 0x7f5034062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.485 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.480+0000 7f5053fff700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 0x7f5034062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5044003a80 tx=0x7f5044007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.491 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.484+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f504c0509b0 con 0x7f50540cba30 2023-10-07T23:30:22.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.652+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50540711c0 con 0x7f50540cba30 2023-10-07T23:30:22.658 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.652+0000 7f5050ff9700 1 -- 172.21.15.57:0/109533414 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15506 (secure 0 0 0) 0x7f504c050540 con 0x7f50540cba30 2023-10-07T23:30:22.659 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:22.659 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":50,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:20.008535+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 msgr2=0x7f5034062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 0x7f5034062a00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5044003a80 tx=0x7f5044007ec0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 msgr2=0x7f505414dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 0x7f505414dd60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f504c00cb10 tx=0x7f504c001970 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 shutdown_connections 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50540cba30 0x7f505414dd60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50540c6870 0x7f50541490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5034060550 0x7f5034062a00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 --2- 172.21.15.57:0/109533414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50540abb30 0x7f5054168cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 >> 172.21.15.57:0/109533414 conn(0x7f50540670e0 msgr2=0x7f50540cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 shutdown_connections 2023-10-07T23:30:22.663 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.656+0000 7f505a7b3700 1 -- 172.21.15.57:0/109533414 wait complete. 2023-10-07T23:30:22.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-07T23:21:13.132066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-07T23:23:16.809985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-07T23:23:17.488606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-07T23:30:22.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-07T23:30:22.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 Processor -- start 2023-10-07T23:30:22.788 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 -- start start 2023-10-07T23:30:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e8815cb50 0x7f2e880ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 0x7f2e880ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e880c9320 con 0x7f2e88158d00 2023-10-07T23:30:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880c8860 con 0x7f2e8815cb50 2023-10-07T23:30:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e88067600 con 0x7f2e881590d0 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f2e8815cb50 0x7f2e880ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8ec80700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 0x7f2e880ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8ec80700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 0x7f2e880ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:42776/0 (socket says 172.21.15.57:42776) 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8ec80700 1 -- 172.21.15.57:0/1810281850 learned_addr learned my addr 172.21.15.57:0/1810281850 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8d47d700 1 -- 172.21.15.57:0/1810281850 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605856372 0 0) 0x7f2e880c9320 con 0x7f2e88158d00 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/1810281850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 msgr2=0x7f2e880ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 --2- 172.21.15.57:0/1810281850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 0x7f2e880ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/1810281850 >> v1:172.21.15.154:6789/0 conn(0x7f2e88158d00 legacy=0x7f2e880c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/1810281850 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880add10 con 0x7f2e8815cb50 2023-10-07T23:30:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8dc7e700 1 --2- 172.21.15.57:0/1810281850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e8815cb50 0x7f2e880ad7d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098e0 tx=0x7f2e780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbab71607a96fbae server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8d47d700 1 -- 172.21.15.57:0/1810281850 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800b640 con 0x7f2e8815cb50 2023-10-07T23:30:22.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8d47d700 1 -- 172.21.15.57:0/1810281850 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f2e7800b7a0 con 0x7f2e8815cb50 2023-10-07T23:30:22.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8d47d700 1 -- 172.21.15.57:0/1810281850 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800bb40 con 0x7f2e8815cb50 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 -- 172.21.15.57:0/1810281850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e8815cb50 msgr2=0x7f2e880ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.784+0000 7f2e8f481700 1 --2- 172.21.15.57:0/1810281850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e8815cb50 0x7f2e880ad7d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098e0 tx=0x7f2e780091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- 172.21.15.57:0/1810281850 shutdown_connections 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 --2- 172.21.15.57:0/1810281850 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e881590d0 0x7f2e880ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 --2- 172.21.15.57:0/1810281850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e8815cb50 0x7f2e880ad7d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- 172.21.15.57:0/1810281850 >> 172.21.15.57:0/1810281850 conn(0x7f2e880692f0 msgr2=0x7f2e880696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- 172.21.15.57:0/1810281850 shutdown_connections 2023-10-07T23:30:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- 172.21.15.57:0/1810281850 wait complete. 2023-10-07T23:30:22.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 Processor -- start 2023-10-07T23:30:22.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- start start 2023-10-07T23:30:22.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e88158d00 0x7f2e88154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 0x7f2e880c1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880c4c60 con 0x7f2e881590d0 2023-10-07T23:30:22.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880c4f00 con 0x7f2e8815cb50 2023-10-07T23:30:22.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8f481700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880c51e0 con 0x7f2e88158d00 2023-10-07T23:30:22.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60890/0 (socket says 172.21.15.57:60890) 2023-10-07T23:30:22.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/2146278653 learned_addr learned my addr 172.21.15.57:0/2146278653 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:22.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8e47f700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e88158d00 0x7f2e88154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8ec80700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 0x7f2e880c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e88158d00 msgr2=0x7f2e88154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e88158d00 0x7f2e88154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 msgr2=0x7f2e880c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 0x7f2e880c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8dc7e700 1 -- 172.21.15.57:0/2146278653 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e78009420 con 0x7f2e881590d0 2023-10-07T23:30:22.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.788+0000 7f2e8ec80700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 0x7f2e880c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e8dc7e700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009f80 tx=0x7f2e780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78000cc0 con 0x7f2e881590d0 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f2e7800b910 con 0x7f2e881590d0 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e880c54c0 con 0x7f2e881590d0 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78025650 con 0x7f2e881590d0 2023-10-07T23:30:22.800 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880c4400 con 0x7f2e881590d0 2023-10-07T23:30:22.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e880c56d0 con 0x7f2e881590d0 2023-10-07T23:30:22.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f2e78023070 con 0x7f2e881590d0 2023-10-07T23:30:22.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 0x7f2e6c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:22.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f2e7807db30 con 0x7f2e881590d0 2023-10-07T23:30:22.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.792+0000 7f2e8e47f700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 0x7f2e6c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:22.803 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.796+0000 7f2e8e47f700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 0x7f2e6c062a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e84003820 tx=0x7f2e84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:22.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.800+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2e7804e140 con 0x7f2e881590d0 2023-10-07T23:30:22.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.964+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e880711c0 con 0x7f2e881590d0 2023-10-07T23:30:22.970 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.964+0000 7f2e7e7fc700 1 -- 172.21.15.57:0/2146278653 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e78015020 con 0x7f2e881590d0 2023-10-07T23:30:22.970 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 msgr2=0x7f2e6c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 0x7f2e6c062a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e84003820 tx=0x7f2e84009040 comp rx=0 tx=0).stop 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 msgr2=0x7f2e88156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009f80 tx=0x7f2e780091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 shutdown_connections 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f2e6c060550 0x7f2e6c062a00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.973 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e8815cb50 0x7f2e880c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2e881590d0 0x7f2e88156a40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 --2- 172.21.15.57:0/2146278653 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2e88158d00 0x7f2e88154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 >> 172.21.15.57:0/2146278653 conn(0x7f2e880692f0 msgr2=0x7f2e8806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 shutdown_connections 2023-10-07T23:30:22.974 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:22.968+0000 7f2e8f481700 1 -- 172.21.15.57:0/2146278653 wait complete. 2023-10-07T23:30:22.986 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-07T23:30:23.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.088+0000 7fcbdf2b8700 1 Processor -- start 2023-10-07T23:30:23.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.088+0000 7fcbdf2b8700 1 -- start start 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd806a900 con 0x7fcbd8159030 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd806a600 con 0x7fcbd806cf10 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b0450 con 0x7fcbd80cad10 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd054700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fcbd8159030 0x7fcbd806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46446/0 (socket says 172.21.15.57:46446) 2023-10-07T23:30:23.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd054700 1 -- 172.21.15.57:0/411169673 learned_addr learned my addr 172.21.15.57:0/411169673 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:23.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd855700 1 --2- 172.21.15.57:0/411169673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692195911 0 0) 0x7fcbd806a900 con 0x7fcbd8159030 2023-10-07T23:30:23.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc4003570 con 0x7fcbd8159030 2023-10-07T23:30:23.100 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598290 0 0) 0x7fcbd806a600 con 0x7fcbd806cf10 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd806a900 con 0x7fcbd806cf10 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd855700 1 -- 172.21.15.57:0/411169673 >> v1:172.21.15.154:6789/0 conn(0x7fcbd806cf10 legacy=0x7fcbd80ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd855700 1 -- 172.21.15.57:0/411169673 >> v1:172.21.15.57:6789/0 conn(0x7fcbd8159030 legacy=0x7fcbd806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd855700 1 -- 172.21.15.57:0/411169673 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80cb350 con 0x7fcbd80cad10 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdd855700 1 --2- 172.21.15.57:0/411169673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd80aff10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009e90 tx=0x7fcbd0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306bdc6930f82651 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000e070 con 0x7fcbd80cad10 2023-10-07T23:30:23.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcbd000bb80 con 0x7fcbd80cad10 2023-10-07T23:30:23.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbcbfff700 1 -- 172.21.15.57:0/411169673 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0012430 con 0x7fcbd80cad10 2023-10-07T23:30:23.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/411169673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 msgr2=0x7fcbd80aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/411169673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd80aff10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009e90 tx=0x7fcbd0009780 comp rx=0 tx=0).stop 2023-10-07T23:30:23.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/411169673 shutdown_connections 2023-10-07T23:30:23.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/411169673 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd80aff10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/411169673 >> 172.21.15.57:0/411169673 conn(0x7fcbd8068650 msgr2=0x7fcbd8068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/411169673 shutdown_connections 2023-10-07T23:30:23.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.092+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/411169673 wait complete. 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 Processor -- start 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 -- start start 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd8156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbd8159030 0x7fcbd8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8156450 con 0x7fcbd806cf10 2023-10-07T23:30:23.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8155b40 con 0x7fcbd8159030 2023-10-07T23:30:23.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdf2b8700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c5120 con 0x7fcbd80cad10 2023-10-07T23:30:23.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60902/0 (socket says 172.21.15.57:60902) 2023-10-07T23:30:23.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 -- 172.21.15.57:0/2422444886 learned_addr learned my addr 172.21.15.57:0/2422444886 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:23.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdc853700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd8156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd855700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbd8159030 0x7fcbd8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 -- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 msgr2=0x7fcbd8156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd8156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 -- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbd8159030 msgr2=0x7fcbd8157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbd8159030 0x7fcbd8157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 -- 172.21.15.57:0/2422444886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009a20 con 0x7fcbd806cf10 2023-10-07T23:30:23.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.096+0000 7fcbdd054700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c7c0 tx=0x7fcbcc007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc005b30 con 0x7fcbd806cf10 2023-10-07T23:30:23.108 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8155e00 con 0x7fcbd806cf10 2023-10-07T23:30:23.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcbcc005cb0 con 0x7fcbd806cf10 2023-10-07T23:30:23.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd80c6270 con 0x7fcbd806cf10 2023-10-07T23:30:23.109 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc007320 con 0x7fcbd806cf10 2023-10-07T23:30:23.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd80647b0 con 0x7fcbd806cf10 2023-10-07T23:30:23.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcbcc01c440 con 0x7fcbd806cf10 2023-10-07T23:30:23.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 0x7fcbbc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7fcbcc07bd10 con 0x7fcbd806cf10 2023-10-07T23:30:23.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbdc853700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 0x7fcbbc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.111 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.100+0000 7fcbdc853700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 0x7fcbbc062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbd81585e0 tx=0x7fcbd4007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.108+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcbcc04c2a0 con 0x7fcbd806cf10 2023-10-07T23:30:23.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbd80caaf0 con 0x7fcbd806cf10 2023-10-07T23:30:23.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbc97fa700 1 -- 172.21.15.57:0/2422444886 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbcc04f8c0 con 0x7fcbd806cf10 2023-10-07T23:30:23.281 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 msgr2=0x7fcbbc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 0x7fcbbc062a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcbd81585e0 tx=0x7fcbd4007d50 comp rx=0 tx=0).stop 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 msgr2=0x7fcbd81568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c7c0 tx=0x7fcbcc007ea0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 shutdown_connections 2023-10-07T23:30:23.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcbbc060550 0x7fcbbc062a00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbd8159030 0x7fcbd8157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbd80cad10 0x7fcbd8156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 --2- 172.21.15.57:0/2422444886 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcbd806cf10 0x7fcbd81568e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 >> 172.21.15.57:0/2422444886 conn(0x7fcbd8068650 msgr2=0x7fcbd8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 shutdown_connections 2023-10-07T23:30:23.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.276+0000 7fcbdf2b8700 1 -- 172.21.15.57:0/2422444886 wait complete. 2023-10-07T23:30:23.296 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-07T23:30:23.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 Processor -- start 2023-10-07T23:30:23.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 -- start start 2023-10-07T23:30:23.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6280c9580 0x7fa6280c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa628159c60 con 0x7fa628155610 2023-10-07T23:30:23.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa628159f40 con 0x7fa6280c9580 2023-10-07T23:30:23.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.400+0000 7fa62d764700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62815a220 con 0x7fa6281596f0 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6280c9580 0x7fa6280c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6280c9580 0x7fa6280c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:54316/0 (socket says 172.21.15.57:54316) 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 -- 172.21.15.57:0/2643266208 learned_addr learned my addr 172.21.15.57:0/2643266208 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:42812/0 (socket says 172.21.15.57:42812) 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 -- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 msgr2=0x7fa6280cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 -- 172.21.15.57:0/2643266208 >> v1:172.21.15.154:6789/0 conn(0x7fa628155610 legacy=0x7fa6281559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 -- 172.21.15.57:0/2643266208 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62815a500 con 0x7fa6280c9580 2023-10-07T23:30:23.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa6277fe700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:23.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62cf63700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6280c9580 0x7fa6280c9a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa62000bc50 tx=0x7fa62000b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26638b780928fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa626ffd700 1 -- 172.21.15.57:0/2643266208 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000dbf0 con 0x7fa6280c9580 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa626ffd700 1 -- 172.21.15.57:0/2643266208 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fa62000dd50 con 0x7fa6280c9580 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa626ffd700 1 -- 172.21.15.57:0/2643266208 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6200196b0 con 0x7fa6280c9580 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 -- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6280c9580 msgr2=0x7fa6280c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6280c9580 0x7fa6280c9a00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa62000bc50 tx=0x7fa62000b4f0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 -- 172.21.15.57:0/2643266208 shutdown_connections 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6280c9580 0x7fa6280c9a00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 --2- 172.21.15.57:0/2643266208 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6281596f0 0x7fa6280cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 -- 172.21.15.57:0/2643266208 >> 172.21.15.57:0/2643266208 conn(0x7fa6280670e0 msgr2=0x7fa6280674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 -- 172.21.15.57:0/2643266208 shutdown_connections 2023-10-07T23:30:23.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.404+0000 7fa62d764700 1 -- 172.21.15.57:0/2643266208 wait complete. 2023-10-07T23:30:23.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 Processor -- start 2023-10-07T23:30:23.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- start start 2023-10-07T23:30:23.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6280c9580 0x7fa6280c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6281596f0 0x7fa6281585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280b1720 con 0x7fa628155610 2023-10-07T23:30:23.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280b06a0 con 0x7fa6281596f0 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6280b0980 con 0x7fa6280c9580 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60904/0 (socket says 172.21.15.57:60904) 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 -- 172.21.15.57:0/3550646380 learned_addr learned my addr 172.21.15.57:0/3550646380 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa627fff700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6280c9580 0x7fa6280c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62cf63700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6281596f0 0x7fa6281585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 -- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6280c9580 msgr2=0x7fa6280c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6280c9580 0x7fa6280c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 -- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6281596f0 msgr2=0x7fa6281585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6281596f0 0x7fa6281585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 -- 172.21.15.57:0/3550646380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618009a20 con 0x7fa628155610 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa6277fe700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c170 tx=0x7fa61800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800cc30 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fa61800f040 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62000b790 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618014480 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.408+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6280b1220 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6280711c0 con 0x7fa628155610 2023-10-07T23:30:23.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa6180147f0 con 0x7fa628155610 2023-10-07T23:30:23.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa60ffff700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 0x7fa614062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7fa61807d600 con 0x7fa628155610 2023-10-07T23:30:23.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa627fff700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 0x7fa614062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.419 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.412+0000 7fa627fff700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 0x7fa614062a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6280cbe10 tx=0x7fa61c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.427 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.420+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa61804daf0 con 0x7fa628155610 2023-10-07T23:30:23.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.584+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6280aac40 con 0x7fa628155610 2023-10-07T23:30:23.589 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.584+0000 7fa60ffff700 1 -- 172.21.15.57:0/3550646380 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa618051110 con 0x7fa628155610 2023-10-07T23:30:23.589 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2023-10-07T23:30:23.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 msgr2=0x7fa614062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 0x7fa614062a30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa6280cbe10 tx=0x7fa61c009040 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 msgr2=0x7fa6280afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c170 tx=0x7fa61800b040 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 shutdown_connections 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa614060580 0x7fa614062a30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6281596f0 0x7fa6281585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa628155610 0x7fa6280afe20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 --2- 172.21.15.57:0/3550646380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa6280c9580 0x7fa6280c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 >> 172.21.15.57:0/3550646380 conn(0x7fa6280670e0 msgr2=0x7fa6280c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 shutdown_connections 2023-10-07T23:30:23.595 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.588+0000 7fa62d764700 1 -- 172.21.15.57:0/3550646380 wait complete. 2023-10-07T23:30:23.610 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:23.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 Processor -- start 2023-10-07T23:30:23.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 -- start start 2023-10-07T23:30:23.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa340157040 0x7fa340155050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.721 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340159560 con 0x7fa340156c70 2023-10-07T23:30:23.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340158950 con 0x7fa3400caed0 2023-10-07T23:30:23.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa346c4d700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340158c30 con 0x7fa340157040 2023-10-07T23:30:23.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa3449e9700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fa340156c70 0x7fa3400ca7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46460/0 (socket says 172.21.15.57:46460) 2023-10-07T23:30:23.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa3449e9700 1 -- 172.21.15.57:0/2493207148 learned_addr learned my addr 172.21.15.57:0/2493207148 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:23.722 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33ffff700 1 --2- 172.21.15.57:0/2493207148 >> v2:172.21.15.154:3300/0 conn(0x7fa340157040 0x7fa340155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420184080 0 0) 0x7fa340159560 con 0x7fa340156c70 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa330003570 con 0x7fa340156c70 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977995994 0 0) 0x7fa330003570 con 0x7fa340156c70 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33ffff700 1 -- 172.21.15.57:0/2493207148 >> v1:172.21.15.154:6789/0 conn(0x7fa3400caed0 legacy=0x7fa340154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33ffff700 1 -- 172.21.15.57:0/2493207148 >> v1:172.21.15.57:6789/0 conn(0x7fa340156c70 legacy=0x7fa3400ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.723 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33ffff700 1 -- 172.21.15.57:0/2493207148 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340158f10 con 0x7fa340157040 2023-10-07T23:30:23.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33ffff700 1 --2- 172.21.15.57:0/2493207148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340157040 0x7fa340155050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0098e0 tx=0x7fa32c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f297c0e54796fdd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334001470 con 0x7fa340157040 2023-10-07T23:30:23.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fa32c011420 con 0x7fa340157040 2023-10-07T23:30:23.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.716+0000 7fa33f7fe700 1 -- 172.21.15.57:0/2493207148 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0117a0 con 0x7fa340157040 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- 172.21.15.57:0/2493207148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340157040 msgr2=0x7fa340155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 --2- 172.21.15.57:0/2493207148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340157040 0x7fa340155050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0098e0 tx=0x7fa32c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- 172.21.15.57:0/2493207148 shutdown_connections 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 --2- 172.21.15.57:0/2493207148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340157040 0x7fa340155050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- 172.21.15.57:0/2493207148 >> 172.21.15.57:0/2493207148 conn(0x7fa3400692f0 msgr2=0x7fa3400696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- 172.21.15.57:0/2493207148 shutdown_connections 2023-10-07T23:30:23.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- 172.21.15.57:0/2493207148 wait complete. 2023-10-07T23:30:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 Processor -- start 2023-10-07T23:30:23.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- start start 2023-10-07T23:30:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400caed0 0x7fa3400b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.729 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa340157040 0x7fa34014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400b1450 con 0x7fa340156c70 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa340167740 con 0x7fa3400caed0 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.720+0000 7fa346c4d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa340167a20 con 0x7fa340157040 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60918/0 (socket says 172.21.15.57:60918) 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 -- 172.21.15.57:0/3929611522 learned_addr learned my addr 172.21.15.57:0/3929611522 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa3449e9700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400caed0 0x7fa3400b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa3451ea700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa340157040 0x7fa34014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 -- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa340157040 msgr2=0x7fa34014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa340157040 0x7fa34014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.730 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 -- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400caed0 msgr2=0x7fa3400b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400caed0 0x7fa3400b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 -- 172.21.15.57:0/3929611522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c009420 con 0x7fa340156c70 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33ffff700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009f80 tx=0x7fa32c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c01cae0 con 0x7fa340156c70 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fa32c01cc60 con 0x7fa340156c70 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa340167d00 con 0x7fa340156c70 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c01a450 con 0x7fa340156c70 2023-10-07T23:30:23.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3401682c0 con 0x7fa340156c70 2023-10-07T23:30:23.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3400711c0 con 0x7fa340156c70 2023-10-07T23:30:23.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.724+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa32c01a7c0 con 0x7fa340156c70 2023-10-07T23:30:23.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.728+0000 7fa33cff9700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 0x7fa324062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:23.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.728+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7fa32c07e220 con 0x7fa340156c70 2023-10-07T23:30:23.732 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.728+0000 7fa3449e9700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 0x7fa324062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:23.733 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.728+0000 7fa3449e9700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 0x7fa324062a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3400c3c30 tx=0x7fa334000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:23.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.732+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa32c04e830 con 0x7fa340156c70 2023-10-07T23:30:23.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.932+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3401591f0 con 0x7fa340156c70 2023-10-07T23:30:23.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa33cff9700 1 -- 172.21.15.57:0/3929611522 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fa32c051e50 con 0x7fa340156c70 2023-10-07T23:30:23.940 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:23.940 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:24:35.918170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:30:23.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 msgr2=0x7fa324062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 0x7fa324062a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3400c3c30 tx=0x7fa334000bc0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 msgr2=0x7fa3400aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009f80 tx=0x7fa32c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 shutdown_connections 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa340157040 0x7fa34014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa340156c70 0x7fa3400aebc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa324060550 0x7fa324062a00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 --2- 172.21.15.57:0/3929611522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400caed0 0x7fa3400b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 >> 172.21.15.57:0/3929611522 conn(0x7fa3400692f0 msgr2=0x7fa34014cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:23.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 shutdown_connections 2023-10-07T23:30:23.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:23.936+0000 7fa346c4d700 1 -- 172.21.15.57:0/3929611522 wait complete. 2023-10-07T23:30:23.944 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2023-10-07T23:30:23.960 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:24.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 Processor -- start 2023-10-07T23:30:24.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 -- start start 2023-10-07T23:30:24.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb5280c6c10 0x7fb5280c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280cac40 con 0x7fb5280cba50 2023-10-07T23:30:24.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280ca030 con 0x7fb5280bee60 2023-10-07T23:30:24.065 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb530733700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5280ca310 con 0x7fb5280c6c10 2023-10-07T23:30:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52dcce700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb5280cba50 0x7fb5280cbe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46466/0 (socket says 172.21.15.57:46466) 2023-10-07T23:30:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52dcce700 1 -- 172.21.15.57:0/2953847522 learned_addr learned my addr 172.21.15.57:0/2953847522 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:24.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52ecd0700 1 --2- 172.21.15.57:0/2953847522 >> v2:172.21.15.57:3300/0 conn(0x7fb5280c6c10 0x7fb5280c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363208610 0 0) 0x7fb5280cac40 con 0x7fb5280cba50 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb510003570 con 0x7fb5280cba50 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934414819 0 0) 0x7fb5280ca030 con 0x7fb5280bee60 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5280cac40 con 0x7fb5280bee60 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52ecd0700 1 -- 172.21.15.57:0/2953847522 >> v1:172.21.15.154:6789/0 conn(0x7fb5280bee60 legacy=0x7fb5280cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52ecd0700 1 -- 172.21.15.57:0/2953847522 >> v1:172.21.15.57:6789/0 conn(0x7fb5280cba50 legacy=0x7fb5280cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2885521741 0 0) 0x7fb510003570 con 0x7fb5280cba50 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52ecd0700 1 -- 172.21.15.57:0/2953847522 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280ca5f0 con 0x7fb5280c6c10 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52ecd0700 1 --2- 172.21.15.57:0/2953847522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280c6c10 0x7fb5280c7000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb5180098e0 tx=0x7fb5180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859e9e8f4aff56f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200017f0 con 0x7fb5280c6c10 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fb518011420 con 0x7fb5280c6c10 2023-10-07T23:30:24.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.060+0000 7fb52d4cd700 1 -- 172.21.15.57:0/2953847522 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180117c0 con 0x7fb5280c6c10 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- 172.21.15.57:0/2953847522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280c6c10 msgr2=0x7fb5280c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 --2- 172.21.15.57:0/2953847522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280c6c10 0x7fb5280c7000 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb5180098e0 tx=0x7fb5180091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- 172.21.15.57:0/2953847522 shutdown_connections 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 --2- 172.21.15.57:0/2953847522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280c6c10 0x7fb5280c7000 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- 172.21.15.57:0/2953847522 >> 172.21.15.57:0/2953847522 conn(0x7fb5280670e0 msgr2=0x7fb5280674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- 172.21.15.57:0/2953847522 shutdown_connections 2023-10-07T23:30:24.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- 172.21.15.57:0/2953847522 wait complete. 2023-10-07T23:30:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 Processor -- start 2023-10-07T23:30:24.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.064+0000 7fb530733700 1 -- start start 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5280bee60 0x7fb5280c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5280c6c10 0x7fb5280b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c3ff0 con 0x7fb5280cba50 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c42d0 con 0x7fb5280c6c10 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c45b0 con 0x7fb5280bee60 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60940/0 (socket says 172.21.15.57:60940) 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 -- 172.21.15.57:0/42428438 learned_addr learned my addr 172.21.15.57:0/42428438 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52dcce700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5280c6c10 0x7fb5280b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.073 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52e4cf700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5280bee60 0x7fb5280c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 -- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5280bee60 msgr2=0x7fb5280c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5280bee60 0x7fb5280c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 -- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5280c6c10 msgr2=0x7fb5280b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5280c6c10 0x7fb5280b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 -- 172.21.15.57:0/42428438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518009420 con 0x7fb5280cba50 2023-10-07T23:30:24.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb52ecd0700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb5180093f0 tx=0x7fb5180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51801cc80 con 0x7fb5280cba50 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fb51801cde0 con 0x7fb5280cba50 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5280c4890 con 0x7fb5280cba50 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51801a6a0 con 0x7fb5280cba50 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5280b1560 con 0x7fb5280cba50 2023-10-07T23:30:24.075 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.068+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5280711c0 con 0x7fb5280cba50 2023-10-07T23:30:24.076 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.072+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb51801aa10 con 0x7fb5280cba50 2023-10-07T23:30:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.072+0000 7fb51e7fc700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.072+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7fb51807e1f0 con 0x7fb5280cba50 2023-10-07T23:30:24.077 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.072+0000 7fb52e4cf700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.078 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.072+0000 7fb52e4cf700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 0x7fb514062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb524007fd0 tx=0x7fb524007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.081 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.076+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb51804e760 con 0x7fb5280cba50 2023-10-07T23:30:24.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.240+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5280ca8d0 con 0x7fb5280cba50 2023-10-07T23:30:24.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.240+0000 7fb51e7fc700 1 -- 172.21.15.57:0/42428438 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15506 (secure 0 0 0) 0x7fb518051d80 con 0x7fb5280cba50 2023-10-07T23:30:24.247 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:24.248 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":50,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:20.008535+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 msgr2=0x7fb514062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 0x7fb514062a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb524007fd0 tx=0x7fb524007d50 comp rx=0 tx=0).stop 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 msgr2=0x7fb5280c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb5180093f0 tx=0x7fb5180091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 shutdown_connections 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb5280cba50 0x7fb5280c2da0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb514060550 0x7fb514062a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5280c6c10 0x7fb5280b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 --2- 172.21.15.57:0/42428438 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb5280bee60 0x7fb5280c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 >> 172.21.15.57:0/42428438 conn(0x7fb5280670e0 msgr2=0x7fb5280aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 shutdown_connections 2023-10-07T23:30:24.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.244+0000 7fb530733700 1 -- 172.21.15.57:0/42428438 wait complete. 2023-10-07T23:30:24.262 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T23:30:24.263 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:24.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 Processor -- start 2023-10-07T23:30:24.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 -- start start 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f0a1406bb00 0x7f0a1415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1415ccb0 con 0x7f0a140aefb0 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1415cf70 con 0x7f0a1406cf10 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1c116700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1415d250 con 0x7f0a1406bb00 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f0a140aefb0 0x7f0a1406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46482/0 (socket says 172.21.15.57:46482) 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 -- 172.21.15.57:0/3198591559 learned_addr learned my addr 172.21.15.57:0/3198591559 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:24.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/3198591559 >> v2:172.21.15.57:3300/0 conn(0x7f0a1406bb00 0x7f0a1415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119017537 0 0) 0x7f0a1415ccb0 con 0x7f0a140aefb0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a00003570 con 0x7f0a140aefb0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1245182511 0 0) 0x7f0a00003570 con 0x7f0a140aefb0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 >> v2:172.21.15.57:3300/0 conn(0x7f0a1406bb00 msgr2=0x7f0a1415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 --2- 172.21.15.57:0/3198591559 >> v2:172.21.15.57:3300/0 conn(0x7f0a1406bb00 0x7f0a1415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 >> v1:172.21.15.154:6789/0 conn(0x7f0a1406cf10 legacy=0x7f0a141599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1415d530 con 0x7f0a140aefb0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f0a10001470 con 0x7f0a140aefb0 2023-10-07T23:30:24.376 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 >> v1:172.21.15.57:6789/0 conn(0x7f0a140aefb0 legacy=0x7f0a1406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 0x7f0a00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a00004810 0x7f0a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a000051c0 0x7f0a0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415d250 con 0x7f0a00003a60 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0000c0d0 con 0x7f0a000051c0 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0000c290 con 0x7f0a00004810 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 0x7f0a00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a000051c0 0x7f0a0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 -- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a00004810 msgr2=0x7f0a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a00004810 0x7f0a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 -- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a000051c0 msgr2=0x7f0a0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a000051c0 0x7f0a0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 -- 172.21.15.57:0/3198591559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00003570 con 0x7f0a00003a60 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a000051c0 0x7f0a0000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a1a6b3700 1 -- 172.21.15.57:0/3198591559 reap_dead start 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 0x7f0a00003e70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c3c0 tx=0x7f0a10001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6d7ae8478d5034 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000cc90 con 0x7f0a00003a60 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f0a1000cdf0 con 0x7f0a00003a60 2023-10-07T23:30:24.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.368+0000 7f0a18eb0700 1 -- 172.21.15.57:0/3198591559 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10005b50 con 0x7f0a00003a60 2023-10-07T23:30:24.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 msgr2=0x7f0a00003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 0x7f0a00003e70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c3c0 tx=0x7f0a10001e50 comp rx=0 tx=0).stop 2023-10-07T23:30:24.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- 172.21.15.57:0/3198591559 shutdown_connections 2023-10-07T23:30:24.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 --2- 172.21.15.57:0/3198591559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a00003a60 0x7f0a00003e70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- 172.21.15.57:0/3198591559 >> 172.21.15.57:0/3198591559 conn(0x7f0a14068630 msgr2=0x7f0a14068a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- 172.21.15.57:0/3198591559 shutdown_connections 2023-10-07T23:30:24.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- 172.21.15.57:0/3198591559 wait complete. 2023-10-07T23:30:24.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 Processor -- start 2023-10-07T23:30:24.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.372+0000 7f0a1c116700 1 -- start start 2023-10-07T23:30:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a00003a60 0x7f0a1416cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a1415bbc0 0x7f0a14155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140c7530 con 0x7f0a1415c030 2023-10-07T23:30:24.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1416dad0 con 0x7f0a00003a60 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1416d510 con 0x7f0a1415bbc0 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60964/0 (socket says 172.21.15.57:60964) 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 -- 172.21.15.57:0/470881709 learned_addr learned my addr 172.21.15.57:0/470881709 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a00003a60 0x7f0a1416cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a196b1700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a1415bbc0 0x7f0a14155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 -- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a1415bbc0 msgr2=0x7f0a14155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a1415bbc0 0x7f0a14155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 -- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a00003a60 msgr2=0x7f0a1416cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a00003a60 0x7f0a1416cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 -- 172.21.15.57:0/470881709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1416d7f0 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1a6b3700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0a140b5fe0 tx=0x7f0a0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c009b70 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f0a0c00fb00 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c00fe80 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a140c71c0 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.376+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1414ce30 con 0x7f0a1415c030 2023-10-07T23:30:24.384 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1414d050 con 0x7f0a1415c030 2023-10-07T23:30:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0a0c016630 con 0x7f0a1415c030 2023-10-07T23:30:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a09ffb700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 0x7f09f8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.385 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f0a0c07d1c0 con 0x7f0a1415c030 2023-10-07T23:30:24.386 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 0x7f09f8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.387 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.380+0000 7f0a19eb2700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 0x7f09f8062a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a10007cd0 tx=0x7f0a10001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.388+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0a1414d050 con 0x7f0a1415c030 2023-10-07T23:30:24.593 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.588+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a1414d050 con 0x7f0a1415c030 2023-10-07T23:30:24.594 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.588+0000 7f0a09ffb700 1 -- 172.21.15.57:0/470881709 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f0a1414d050 con 0x7f0a1415c030 2023-10-07T23:30:24.595 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:24.595 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:24:35.918170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:30:24.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 msgr2=0x7f09f8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 0x7f09f8062a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a10007cd0 tx=0x7f0a10001e50 comp rx=0 tx=0).stop 2023-10-07T23:30:24.597 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 msgr2=0x7f0a14156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0a140b5fe0 tx=0x7f0a0c00b040 comp rx=0 tx=0).stop 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 shutdown_connections 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0a1415c030 0x7f0a14156520 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0a1415bbc0 0x7f0a14155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f09f8060550 0x7f09f8062a00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 --2- 172.21.15.57:0/470881709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a00003a60 0x7f0a1416cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 >> 172.21.15.57:0/470881709 conn(0x7f0a14068630 msgr2=0x7f0a1414da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 shutdown_connections 2023-10-07T23:30:24.598 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.592+0000 7f0a1c116700 1 -- 172.21.15.57:0/470881709 wait complete. 2023-10-07T23:30:24.599 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2023-10-07T23:30:24.610 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:24.713 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 Processor -- start 2023-10-07T23:30:24.714 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 -- start start 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd1a8068670 0x7fd1a8152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a80c7600 con 0x7fd1a80ca1a0 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a8153000 con 0x7fd1a8159870 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1aed7e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80c69f0 con 0x7fd1a8068670 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1ad31b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd1a80ca1a0 0x7fd1a80ca640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46494/0 (socket says 172.21.15.57:46494) 2023-10-07T23:30:24.715 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1ad31b700 1 -- 172.21.15.57:0/3802293751 learned_addr learned my addr 172.21.15.57:0/3802293751 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3802293751 >> v2:172.21.15.57:3300/0 conn(0x7fd1a8068670 0x7fd1a8152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1a77fe700 1 -- 172.21.15.57:0/3802293751 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823408553 0 0) 0x7fd1a80c7600 con 0x7fd1a80ca1a0 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1a77fe700 1 -- 172.21.15.57:0/3802293751 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd190003570 con 0x7fd1a80ca1a0 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1acb1a700 1 -- 172.21.15.57:0/3802293751 >> v1:172.21.15.154:6789/0 conn(0x7fd1a8159870 legacy=0x7fd1a8156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1acb1a700 1 -- 172.21.15.57:0/3802293751 >> v1:172.21.15.57:6789/0 conn(0x7fd1a80ca1a0 legacy=0x7fd1a80ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.716 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1acb1a700 1 -- 172.21.15.57:0/3802293751 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80c6cd0 con 0x7fd1a8068670 2023-10-07T23:30:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.708+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3802293751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8068670 0x7fd1a8152ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0098e0 tx=0x7fd19c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e46ef780d37580 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.717 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1a77fe700 1 -- 172.21.15.57:0/3802293751 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00b640 con 0x7fd1a8068670 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1a77fe700 1 -- 172.21.15.57:0/3802293751 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd19c00b7a0 con 0x7fd1a8068670 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1a77fe700 1 -- 172.21.15.57:0/3802293751 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00bb40 con 0x7fd1a8068670 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3802293751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8068670 msgr2=0x7fd1a8152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3802293751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8068670 0x7fd1a8152ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0098e0 tx=0x7fd19c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3802293751 shutdown_connections 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3802293751 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8068670 0x7fd1a8152ac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3802293751 >> 172.21.15.57:0/3802293751 conn(0x7fd1a80670e0 msgr2=0x7fd1a80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3802293751 shutdown_connections 2023-10-07T23:30:24.718 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.712+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3802293751 wait complete. 2023-10-07T23:30:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 Processor -- start 2023-10-07T23:30:24.720 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 -- start start 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 0x7fd1a8148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca1a0 0x7fd1a8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8148dc0 con 0x7fd1a8159870 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80b0d30 con 0x7fd1a8068670 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a806b2d0 con 0x7fd1a80ca1a0 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.724 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60978/0 (socket says 172.21.15.57:60978) 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1a7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca1a0 0x7fd1a8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 -- 172.21.15.57:0/3827020212 learned_addr learned my addr 172.21.15.57:0/3827020212 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 0x7fd1a8148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 -- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca1a0 msgr2=0x7fd1a8151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca1a0 0x7fd1a8151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 -- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 msgr2=0x7fd1a8148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 0x7fd1a8148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 -- 172.21.15.57:0/3827020212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c009420 con 0x7fd1a8159870 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 0x7fd1a8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1ad31b700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0001460 tx=0x7fd1a0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.725 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000a940 con 0x7fd1a8159870 2023-10-07T23:30:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fd1a000aaa0 con 0x7fd1a8159870 2023-10-07T23:30:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a806b5b0 con 0x7fd1a8159870 2023-10-07T23:30:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.716+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000ae20 con 0x7fd1a8159870 2023-10-07T23:30:24.726 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a806bb70 con 0x7fd1a8159870 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a80711c0 con 0x7fd1a8159870 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd1a0007300 con 0x7fd1a8159870 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1a4ff9700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 0x7fd194062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7fd1a007cca0 con 0x7fd1a8159870 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 0x7fd194062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:24.727 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.720+0000 7fd1acb1a700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 0x7fd194062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00a010 tx=0x7fd19c0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:24.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.728+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd1a0081070 con 0x7fd1a8159870 2023-10-07T23:30:24.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.928+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a80c6fb0 con 0x7fd1a8159870 2023-10-07T23:30:24.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.928+0000 7fd1a4ff9700 1 -- 172.21.15.57:0/3827020212 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fd1a004d300 con 0x7fd1a8159870 2023-10-07T23:30:24.934 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:24.934 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-07T23:23:18.503008+0000","modified":"2023-10-07T23:24:35.918170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5359":{"gid":5359,"name":"d","rank":0,"incarnation":17,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6835/320776638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":320776638},{"type":"v1","addr":"172.21.15.154:6835","nonce":320776638}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 msgr2=0x7fd194062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 0x7fd194062a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00a010 tx=0x7fd19c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 msgr2=0x7fd1a8151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0001460 tx=0x7fd1a0001ae0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 shutdown_connections 2023-10-07T23:30:24.937 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1a8159870 0x7fd1a8151f90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd1a80ca1a0 0x7fd1a8151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd194060550 0x7fd194062a00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 --2- 172.21.15.57:0/3827020212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1a8068670 0x7fd1a8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 >> 172.21.15.57:0/3827020212 conn(0x7fd1a80670e0 msgr2=0x7fd1a80c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 shutdown_connections 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:24.932+0000 7fd1aed7e700 1 -- 172.21.15.57:0/3827020212 wait complete. 2023-10-07T23:30:24.938 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 20 2023-10-07T23:30:24.949 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:25.058 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 Processor -- start 2023-10-07T23:30:25.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 -- start start 2023-10-07T23:30:25.059 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f91dc151090 0x7f91dc15a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc06ae80 con 0x7f91dc0c8920 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc15b050 con 0x7f91dc157d90 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91e0890700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc15b330 con 0x7f91dc151090 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.052+0000 7f91db59e700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f91dc0c8920 0x7f91dc06b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46496/0 (socket says 172.21.15.57:46496) 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91db59e700 1 -- 172.21.15.57:0/3513569961 learned_addr learned my addr 172.21.15.57:0/3513569961 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:25.060 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91dad9d700 1 --2- 172.21.15.57:0/3513569961 >> v2:172.21.15.57:3300/0 conn(0x7f91dc151090 0x7f91dc15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91dad9d700 1 -- 172.21.15.57:0/3513569961 >> v1:172.21.15.154:6789/0 conn(0x7f91dc157d90 legacy=0x7f91dc0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91dad9d700 1 -- 172.21.15.57:0/3513569961 >> v1:172.21.15.57:6789/0 conn(0x7f91dc0c8920 legacy=0x7f91dc06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91dad9d700 1 -- 172.21.15.57:0/3513569961 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc06ab10 con 0x7f91dc151090 2023-10-07T23:30:25.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91dad9d700 1 --2- 172.21.15.57:0/3513569961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc15a750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098e0 tx=0x7f91cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2e96b4c89846f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.061 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91da59c700 1 -- 172.21.15.57:0/3513569961 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009e80 con 0x7f91dc151090 2023-10-07T23:30:25.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91da59c700 1 -- 172.21.15.57:0/3513569961 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f91cc00b860 con 0x7f91dc151090 2023-10-07T23:30:25.062 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91da59c700 1 -- 172.21.15.57:0/3513569961 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00bbe0 con 0x7f91dc151090 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 -- 172.21.15.57:0/3513569961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 msgr2=0x7f91dc15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 --2- 172.21.15.57:0/3513569961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc15a750 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0098e0 tx=0x7f91cc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 -- 172.21.15.57:0/3513569961 shutdown_connections 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 --2- 172.21.15.57:0/3513569961 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc15a750 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 -- 172.21.15.57:0/3513569961 >> 172.21.15.57:0/3513569961 conn(0x7f91dc0670e0 msgr2=0x7f91dc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 -- 172.21.15.57:0/3513569961 shutdown_connections 2023-10-07T23:30:25.063 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.056+0000 7f91e0890700 1 -- 172.21.15.57:0/3513569961 wait complete. 2023-10-07T23:30:25.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 Processor -- start 2023-10-07T23:30:25.064 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- start start 2023-10-07T23:30:25.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 0x7f91dc144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.066 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91dc157d90 0x7f91dc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0c94c0 con 0x7f91dc151090 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0c5350 con 0x7f91dc0c8920 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0c5630 con 0x7f91dc157d90 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:60996/0 (socket says 172.21.15.57:60996) 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 -- 172.21.15.57:0/2987907294 learned_addr learned my addr 172.21.15.57:0/2987907294 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91db59e700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91dc157d90 0x7f91dc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91dad9d700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 0x7f91dc144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 -- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91dc157d90 msgr2=0x7f91dc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.067 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91dc157d90 0x7f91dc155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 -- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 msgr2=0x7f91dc144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 0x7f91dc144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 -- 172.21.15.57:0/2987907294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009420 con 0x7f91dc151090 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91dad9d700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 0x7f91dc144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d3fff700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c590 tx=0x7f91c400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011590 con 0x7f91dc151090 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f91c40116f0 con 0x7f91dc151090 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc0c5910 con 0x7f91dc151090 2023-10-07T23:30:25.068 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4011ab0 con 0x7f91dc151090 2023-10-07T23:30:25.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.060+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc144840 con 0x7f91dc151090 2023-10-07T23:30:25.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc0711c0 con 0x7f91dc151090 2023-10-07T23:30:25.069 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f91c401b3b0 con 0x7f91dc151090 2023-10-07T23:30:25.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91d2ffd700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 0x7f91bc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f91c407be40 con 0x7f91dc151090 2023-10-07T23:30:25.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91dad9d700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 0x7f91bc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.070 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.064+0000 7f91dad9d700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 0x7f91bc062a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0093c0 tx=0x7f91cc0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.074 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.068+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f91c404c3b0 con 0x7f91dc151090 2023-10-07T23:30:25.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.232+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91dc158320 con 0x7f91dc151090 2023-10-07T23:30:25.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.232+0000 7f91d2ffd700 1 -- 172.21.15.57:0/2987907294 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15506 (secure 0 0 0) 0x7f91c4017070 con 0x7f91dc151090 2023-10-07T23:30:25.239 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:25.240 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":50,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:20.008535+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-07T23:23:16.809985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-07T23:23:17.488606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.37","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 msgr2=0x7f91bc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 0x7f91bc062a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0093c0 tx=0x7f91cc0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 msgr2=0x7f91dc0c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f91c400c590 tx=0x7f91c400c950 comp rx=0 tx=0).stop 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 shutdown_connections 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f91dc157d90 0x7f91dc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91dc151090 0x7f91dc0c48e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f91bc060580 0x7f91bc062a30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 --2- 172.21.15.57:0/2987907294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f91dc0c8920 0x7f91dc144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 >> 172.21.15.57:0/2987907294 conn(0x7f91dc0670e0 msgr2=0x7f91dc0c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:25.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 shutdown_connections 2023-10-07T23:30:25.243 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.236+0000 7f91e0890700 1 -- 172.21.15.57:0/2987907294 wait complete. 2023-10-07T23:30:25.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-07T23:30:25.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.360+0000 7f7c6a228700 1 Processor -- start 2023-10-07T23:30:25.367 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.360+0000 7f7c6a228700 1 -- start start 2023-10-07T23:30:25.368 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c6406cf10 0x7f7c64157080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c640cbaa0 0x7f7c640ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c640cca30 con 0x7f7c64156cb0 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640ae5a0 con 0x7f7c6406cf10 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6406aa90 con 0x7f7c640cbaa0 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7c6406cf10 0x7f7c64157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c637fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f7c64156cb0 0x7f7c6406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:46510/0 (socket says 172.21.15.57:46510) 2023-10-07T23:30:25.369 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c637fe700 1 -- 172.21.15.57:0/2405043979 learned_addr learned my addr 172.21.15.57:0/2405043979 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:25.370 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c63fff700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c640cbaa0 0x7f7c640ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 -- 172.21.15.57:0/2405043979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c640cbaa0 msgr2=0x7f7c640ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:25.371 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c640cbaa0 0x7f7c640ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 -- 172.21.15.57:0/2405043979 >> v1:172.21.15.57:6789/0 conn(0x7f7c64156cb0 legacy=0x7f7c6406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c627fc700 1 -- 172.21.15.57:0/2405043979 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617759597 0 0) 0x7f7c640cca30 con 0x7f7c64156cb0 2023-10-07T23:30:25.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 -- 172.21.15.57:0/2405043979 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640cbf00 con 0x7f7c6406cf10 2023-10-07T23:30:25.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c62ffd700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c6406cf10 0x7f7c64157080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098e0 tx=0x7f7c540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da64e97bc4a9625a server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.372 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c627fc700 1 -- 172.21.15.57:0/2405043979 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400b640 con 0x7f7c6406cf10 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c627fc700 1 -- 172.21.15.57:0/2405043979 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f7c5400b7a0 con 0x7f7c6406cf10 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c627fc700 1 -- 172.21.15.57:0/2405043979 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400bb40 con 0x7f7c6406cf10 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- 172.21.15.57:0/2405043979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c6406cf10 msgr2=0x7f7c64157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c6406cf10 0x7f7c64157080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098e0 tx=0x7f7c540091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- 172.21.15.57:0/2405043979 shutdown_connections 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c640cbaa0 0x7f7c640ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.373 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 --2- 172.21.15.57:0/2405043979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c6406cf10 0x7f7c64157080 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- 172.21.15.57:0/2405043979 >> 172.21.15.57:0/2405043979 conn(0x7f7c64062ba0 msgr2=0x7f7c64062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:25.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- 172.21.15.57:0/2405043979 shutdown_connections 2023-10-07T23:30:25.374 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.364+0000 7f7c6a228700 1 -- 172.21.15.57:0/2405043979 wait complete. 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 Processor -- start 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 -- start start 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c6406cf10 0x7f7c640c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c64156cb0 0x7f7c641544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.377 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640c4ae0 con 0x7f7c640cbaa0 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6416ad30 con 0x7f7c6406cf10 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c6a228700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6416b1c0 con 0x7f7c64156cb0 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32780/0 (socket says 172.21.15.57:32780) 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 -- 172.21.15.57:0/765110786 learned_addr learned my addr 172.21.15.57:0/765110786 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c63fff700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c64156cb0 0x7f7c641544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c637fe700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c6406cf10 0x7f7c640c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 -- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c64156cb0 msgr2=0x7f7c641544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.378 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c64156cb0 0x7f7c641544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 -- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c6406cf10 msgr2=0x7f7c640c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c6406cf10 0x7f7c640c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.368+0000 7f7c62ffd700 1 -- 172.21.15.57:0/765110786 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54009420 con 0x7f7c640cbaa0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c62ffd700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c640c9280 tx=0x7f7c540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400b910 con 0x7f7c640cbaa0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6416a400 con 0x7f7c640cbaa0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f7c5400ba70 con 0x7f7c640cbaa0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c54011650 con 0x7f7c640cbaa0 2023-10-07T23:30:25.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6416a9c0 con 0x7f7c640cbaa0 2023-10-07T23:30:25.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640647b0 con 0x7f7c640cbaa0 2023-10-07T23:30:25.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.372+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7c54011a80 con 0x7f7c640cbaa0 2023-10-07T23:30:25.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.376+0000 7f7c5b7fe700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 0x7f7c44062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.376+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5806+0+0 (secure 0 0 0) 0x7f7c5407df10 con 0x7f7c640cbaa0 2023-10-07T23:30:25.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.376+0000 7f7c637fe700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 0x7f7c44062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.381 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.376+0000 7f7c637fe700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 0x7f7c44062a00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c007fd0 tx=0x7f7c4c007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.388 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.384+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7c5404e4a0 con 0x7f7c640cbaa0 2023-10-07T23:30:25.588 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.584+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7c640cc190 con 0x7f7c640cbaa0 2023-10-07T23:30:25.588 INFO:tasks.ceph.mon.a.smithi057.stderr:2023-10-07T23:30:25.584+0000 7f9fa093f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-07T23:30:25.599 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c5b7fe700 1 -- 172.21.15.57:0/765110786 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c54051ac0 con 0x7f7c640cbaa0 2023-10-07T23:30:25.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 msgr2=0x7f7c44062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.602 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 0x7f7c44062a00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c007fd0 tx=0x7f7c4c007d30 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 msgr2=0x7f7c64153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7c640c9280 tx=0x7f7c540091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 shutdown_connections 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7c64156cb0 0x7f7c641544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7c640cbaa0 0x7f7c64153fa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7c44060550 0x7f7c44062a00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 --2- 172.21.15.57:0/765110786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c6406cf10 0x7f7c640c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 >> 172.21.15.57:0/765110786 conn(0x7f7c64062ba0 msgr2=0x7f7c640ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 shutdown_connections 2023-10-07T23:30:25.603 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.596+0000 7f7c6a228700 1 -- 172.21.15.57:0/765110786 wait complete. 2023-10-07T23:30:25.604 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-07T23:30:25.616 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-07T23:30:25.643 INFO:tasks.ceph.mds.d.smithi154.stdout:starting mds.d at 2023-10-07T23:30:25.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.724+0000 7fcd26699700 1 Processor -- start 2023-10-07T23:30:25.731 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.724+0000 7fcd26699700 1 -- start start 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcd2014f9f0 0x7fcd2014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd200cc1a0 0x7fcd200c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd200c86f0 con 0x7fcd200c8290 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd20155c70 con 0x7fcd2014f9f0 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd20155f30 con 0x7fcd200cc1a0 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcd2014f9f0 0x7fcd2014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fcd2014f9f0 0x7fcd2014fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32782/0 (socket says 172.21.15.57:32782) 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 -- 172.21.15.57:0/3200126392 learned_addr learned my addr 172.21.15.57:0/3200126392 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:25.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd24c36700 1 --2- 172.21.15.57:0/3200126392 >> v2:172.21.15.154:3300/0 conn(0x7fcd200cc1a0 0x7fcd200c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 -- 172.21.15.57:0/3200126392 >> v2:172.21.15.154:3300/0 conn(0x7fcd200cc1a0 msgr2=0x7fcd200c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:25.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 --2- 172.21.15.57:0/3200126392 >> v2:172.21.15.154:3300/0 conn(0x7fcd200cc1a0 0x7fcd200c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 -- 172.21.15.57:0/3200126392 >> v1:172.21.15.154:6789/0 conn(0x7fcd200c8290 legacy=0x7fcd200cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 -- 172.21.15.57:0/3200126392 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20156210 con 0x7fcd2014f9f0 2023-10-07T23:30:25.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1ffff700 1 --2- 172.21.15.57:0/3200126392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd2014f9f0 0x7fcd2014fdc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd200abaf0 tx=0x7fcd14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1988be565b2aacb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1effd700 1 -- 172.21.15.57:0/3200126392 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14009e70 con 0x7fcd2014f9f0 2023-10-07T23:30:25.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1effd700 1 -- 172.21.15.57:0/3200126392 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcd1400b860 con 0x7fcd2014f9f0 2023-10-07T23:30:25.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd1effd700 1 -- 172.21.15.57:0/3200126392 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400bc00 con 0x7fcd2014f9f0 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- 172.21.15.57:0/3200126392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd2014f9f0 msgr2=0x7fcd2014fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 --2- 172.21.15.57:0/3200126392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd2014f9f0 0x7fcd2014fdc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd200abaf0 tx=0x7fcd14009250 comp rx=0 tx=0).stop 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- 172.21.15.57:0/3200126392 shutdown_connections 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 --2- 172.21.15.57:0/3200126392 >> v2:172.21.15.154:3300/0 conn(0x7fcd200cc1a0 0x7fcd200c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 --2- 172.21.15.57:0/3200126392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd2014f9f0 0x7fcd2014fdc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- 172.21.15.57:0/3200126392 >> 172.21.15.57:0/3200126392 conn(0x7fcd20062ba0 msgr2=0x7fcd20062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- 172.21.15.57:0/3200126392 shutdown_connections 2023-10-07T23:30:25.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.728+0000 7fcd26699700 1 -- 172.21.15.57:0/3200126392 wait complete. 2023-10-07T23:30:25.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 Processor -- start 2023-10-07T23:30:25.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 -- start start 2023-10-07T23:30:25.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd200cc1a0 0x7fcd20145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 0x7fcd20157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd201456d0 con 0x7fcd200c8290 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2014c970 con 0x7fcd2014f9f0 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd26699700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20145a90 con 0x7fcd200cc1a0 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:32792/0 (socket says 172.21.15.57:32792) 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 -- 172.21.15.57:0/4205540213 learned_addr learned my addr 172.21.15.57:0/4205540213 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1f7fe700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd200cc1a0 0x7fcd20145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd24c36700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 0x7fcd20157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 -- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd200cc1a0 msgr2=0x7fcd20145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd200cc1a0 0x7fcd20145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 -- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 msgr2=0x7fcd20157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 0x7fcd20157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 -- 172.21.15.57:0/4205540213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140094f0 con 0x7fcd200c8290 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd24c36700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 0x7fcd20157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.732+0000 7fcd1ffff700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd140090b0 tx=0x7fcd14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400bbe0 con 0x7fcd200c8290 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fcd1401f070 con 0x7fcd200c8290 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd200af020 con 0x7fcd200c8290 2023-10-07T23:30:25.742 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140266b0 con 0x7fcd200c8290 2023-10-07T23:30:25.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd200af5e0 con 0x7fcd200c8290 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fcd14026a20 con 0x7fcd200c8290 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200647b0 con 0x7fcd200c8290 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 0x7fcd0c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.736+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5892+0+0 (secure 0 0 0) 0x7fcd14021070 con 0x7fcd200c8290 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.740+0000 7fcd1f7fe700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 0x7fcd0c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:25.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.740+0000 7fcd1f7fe700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 0x7fcd0c062a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd2014f9c0 tx=0x7fcd10001670 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:25.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.744+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd1404eb20 con 0x7fcd200c8290 2023-10-07T23:30:25.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:25.940+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcd2006aa30 con 0x7fcd200c8290 2023-10-07T23:30:26.600 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.596+0000 7fcd07fff700 1 -- 172.21.15.57:0/4205540213 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcd14052140 con 0x7fcd200c8290 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 msgr2=0x7fcd0c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 0x7fcd0c062a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd2014f9c0 tx=0x7fcd10001670 comp rx=0 tx=0).stop 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 msgr2=0x7fcd200ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd140090b0 tx=0x7fcd14009250 comp rx=0 tx=0).stop 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 shutdown_connections 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fcd0c060550 0x7fcd0c062a00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd2014f9f0 0x7fcd20157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcd200cc1a0 0x7fcd20145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 --2- 172.21.15.57:0/4205540213 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd200c8290 0x7fcd200ae990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 >> 172.21.15.57:0/4205540213 conn(0x7fcd20062ba0 msgr2=0x7fcd2006ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:26.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 shutdown_connections 2023-10-07T23:30:26.607 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.600+0000 7fcd26699700 1 -- 172.21.15.57:0/4205540213 wait complete. 2023-10-07T23:30:26.622 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-07T23:30:26.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.736+0000 7f908f4ef700 1 Processor -- start 2023-10-07T23:30:26.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- start start 2023-10-07T23:30:26.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f908814f9f0 0x7f908814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880c8290 0x7f90880cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90880c86f0 con 0x7f90880cbfd0 2023-10-07T23:30:26.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908806cf10 con 0x7f908814f9f0 2023-10-07T23:30:26.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880cc410 con 0x7f90880c8290 2023-10-07T23:30:26.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f908814f9f0 0x7f908814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f908814f9f0 0x7f908814fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40674/0 (socket says 172.21.15.57:40674) 2023-10-07T23:30:26.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 -- 172.21.15.57:0/1832771366 learned_addr learned my addr 172.21.15.57:0/1832771366 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:26.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908ca8a700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880c8290 0x7f90880cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 -- 172.21.15.57:0/1832771366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880c8290 msgr2=0x7f90880cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:26.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880c8290 0x7f90880cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 -- 172.21.15.57:0/1832771366 >> v1:172.21.15.154:6789/0 conn(0x7f90880cbfd0 legacy=0x7f908806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 -- 172.21.15.57:0/1832771366 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088155c70 con 0x7f908814f9f0 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908d28b700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908814f9f0 0x7f908814fdc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90840099b0 tx=0x7f9084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84034941d79b8f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f907ffff700 1 -- 172.21.15.57:0/1832771366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084009ea0 con 0x7f908814f9f0 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f907ffff700 1 -- 172.21.15.57:0/1832771366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f908400b860 con 0x7f908814f9f0 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f907ffff700 1 -- 172.21.15.57:0/1832771366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908400bc00 con 0x7f908814f9f0 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- 172.21.15.57:0/1832771366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908814f9f0 msgr2=0x7f908814fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908814f9f0 0x7f908814fdc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90840099b0 tx=0x7f9084009250 comp rx=0 tx=0).stop 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- 172.21.15.57:0/1832771366 shutdown_connections 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880c8290 0x7f90880cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 --2- 172.21.15.57:0/1832771366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f908814f9f0 0x7f908814fdc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- 172.21.15.57:0/1832771366 >> 172.21.15.57:0/1832771366 conn(0x7f9088062ba0 msgr2=0x7f9088062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:26.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- 172.21.15.57:0/1832771366 shutdown_connections 2023-10-07T23:30:26.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.740+0000 7f908f4ef700 1 -- 172.21.15.57:0/1832771366 wait complete. 2023-10-07T23:30:26.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 Processor -- start 2023-10-07T23:30:26.749 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 -- start start 2023-10-07T23:30:26.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880cbfd0 0x7f9088160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f908814f9f0 0x7f9088160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088152560 con 0x7f90880c8290 2023-10-07T23:30:26.751 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088152840 con 0x7f908814f9f0 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908f4ef700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9088152b20 con 0x7f90880cbfd0 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40684/0 (socket says 172.21.15.57:40684) 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 -- 172.21.15.57:0/2538715092 learned_addr learned my addr 172.21.15.57:0/2538715092 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908ca8a700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880cbfd0 0x7f9088160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.752 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908da8c700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f908814f9f0 0x7f9088160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 -- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880cbfd0 msgr2=0x7f9088160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880cbfd0 0x7f9088160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 -- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f908814f9f0 msgr2=0x7f9088160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:26.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f908814f9f0 0x7f9088160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:26.753 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.744+0000 7f908d28b700 1 -- 172.21.15.57:0/2538715092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840094f0 con 0x7f90880c8290 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f908d28b700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f9084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084000d10 con 0x7f90880c8290 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f908401f070 con 0x7f90880c8290 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9088152e00 con 0x7f90880c8290 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840266d0 con 0x7f90880c8290 2023-10-07T23:30:26.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908814f5a0 con 0x7f90880c8290 2023-10-07T23:30:26.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90880ae050 con 0x7f90880c8290 2023-10-07T23:30:26.755 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.748+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f9084026a40 con 0x7f90880c8290 2023-10-07T23:30:26.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.752+0000 7f907d7fa700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 0x7f906c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:26.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.752+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5892+0+0 (secure 0 0 0) 0x7f9084021070 con 0x7f90880c8290 2023-10-07T23:30:26.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.752+0000 7f908ca8a700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 0x7f906c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:26.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.752+0000 7f908ca8a700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 0x7f906c062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9088150ef0 tx=0x7f90780091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:26.762 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.756+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f90880ae050 con 0x7f90880c8290 2023-10-07T23:30:26.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:26.920+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90880ae050 con 0x7f90880c8290 2023-10-07T23:30:27.606 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.600+0000 7f907d7fa700 1 -- 172.21.15.57:0/2538715092 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7f90880ae050 con 0x7f90880c8290 2023-10-07T23:30:27.612 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 msgr2=0x7f906c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 0x7f906c062a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9088150ef0 tx=0x7f90780091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 msgr2=0x7f908814f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9084006fd0 tx=0x7f9084009250 comp rx=0 tx=0).stop 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 shutdown_connections 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f906c060550 0x7f906c062a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f908814f9f0 0x7f9088160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f90880cbfd0 0x7f9088160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.613 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 --2- 172.21.15.57:0/2538715092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90880c8290 0x7f908814f060 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 >> 172.21.15.57:0/2538715092 conn(0x7f9088062ba0 msgr2=0x7f90880cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:27.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 shutdown_connections 2023-10-07T23:30:27.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.608+0000 7f908f4ef700 1 -- 172.21.15.57:0/2538715092 wait complete. 2023-10-07T23:30:27.614 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2023-10-07T23:30:27.626 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-07T23:30:27.734 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 Processor -- start 2023-10-07T23:30:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 -- start start 2023-10-07T23:30:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 0x7fba380aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3806a900 con 0x7fba380ca600 2023-10-07T23:30:27.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3806a600 con 0x7fba38159030 2023-10-07T23:30:27.736 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.728+0000 7fba3ceb4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b0450 con 0x7fba3806cf10 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba36d9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fba380ca600 0x7fba380caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38830/0 (socket says 172.21.15.57:38830) 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba36d9d700 1 -- 172.21.15.57:0/4026494799 learned_addr learned my addr 172.21.15.57:0/4026494799 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba2dd9b700 1 --2- 172.21.15.57:0/4026494799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 0x7fba380aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992428221 0 0) 0x7fba3806a900 con 0x7fba380ca600 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba24003570 con 0x7fba380ca600 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474616731 0 0) 0x7fba3806a600 con 0x7fba38159030 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3806a900 con 0x7fba38159030 2023-10-07T23:30:27.737 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba2dd9b700 1 -- 172.21.15.57:0/4026494799 >> v1:172.21.15.154:6789/0 conn(0x7fba38159030 legacy=0x7fba3806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba2dd9b700 1 -- 172.21.15.57:0/4026494799 >> v1:172.21.15.57:6789/0 conn(0x7fba380ca600 legacy=0x7fba380caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba2dd9b700 1 -- 172.21.15.57:0/4026494799 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380cb350 con 0x7fba3806cf10 2023-10-07T23:30:27.738 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba2dd9b700 1 --2- 172.21.15.57:0/4026494799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 0x7fba380aff10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba20009910 tx=0x7fba200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0b7d3312649d51 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011630 con 0x7fba3806cf10 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fba20011790 con 0x7fba3806cf10 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba35d9b700 1 -- 172.21.15.57:0/4026494799 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011b10 con 0x7fba3806cf10 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 -- 172.21.15.57:0/4026494799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 msgr2=0x7fba380aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/4026494799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 0x7fba380aff10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fba20009910 tx=0x7fba200091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 -- 172.21.15.57:0/4026494799 shutdown_connections 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/4026494799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba3806cf10 0x7fba380aff10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.739 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 -- 172.21.15.57:0/4026494799 >> 172.21.15.57:0/4026494799 conn(0x7fba38068650 msgr2=0x7fba38068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:27.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 -- 172.21.15.57:0/4026494799 shutdown_connections 2023-10-07T23:30:27.740 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.732+0000 7fba3ceb4700 1 -- 172.21.15.57:0/4026494799 wait complete. 2023-10-07T23:30:27.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 Processor -- start 2023-10-07T23:30:27.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 -- start start 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3806cf10 0x7fba38158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 0x7fba3814f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3815add0 con 0x7fba380ca600 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba38150310 con 0x7fba3806cf10 2023-10-07T23:30:27.744 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3ceb4700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b3800 con 0x7fba38159030 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40690/0 (socket says 172.21.15.57:40690) 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 -- 172.21.15.57:0/1820543624 learned_addr learned my addr 172.21.15.57:0/1820543624 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba36d9d700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 0x7fba3814f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba3659c700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3806cf10 0x7fba38158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 -- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 msgr2=0x7fba3814f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 0x7fba3814f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 -- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3806cf10 msgr2=0x7fba38158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3806cf10 0x7fba38158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba2dd9b700 1 -- 172.21.15.57:0/1820543624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20009420 con 0x7fba380ca600 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.736+0000 7fba36d9d700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 0x7fba3814f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:27.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2dd9b700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fba20009f00 tx=0x7fba200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011900 con 0x7fba380ca600 2023-10-07T23:30:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7fba20011a60 con 0x7fba380ca600 2023-10-07T23:30:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20025620 con 0x7fba380ca600 2023-10-07T23:30:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba380b2ed0 con 0x7fba380ca600 2023-10-07T23:30:27.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba380b3490 con 0x7fba380ca600 2023-10-07T23:30:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba380b3990 con 0x7fba380ca600 2023-10-07T23:30:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fba20025a10 con 0x7fba380ca600 2023-10-07T23:30:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 0x7fba18062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5454+0+0 (secure 0 0 0) 0x7fba2007e2b0 con 0x7fba380ca600 2023-10-07T23:30:27.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.740+0000 7fba3659c700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 0x7fba18062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:27.748 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.744+0000 7fba3659c700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 0x7fba18062a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba28007fd0 tx=0x7fba28007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:27.750 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.744+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fba2004ea00 con 0x7fba380ca600 2023-10-07T23:30:27.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:27.904+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba380647b0 con 0x7fba380ca600 2023-10-07T23:30:28.617 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.608+0000 7fba2effd700 1 -- 172.21.15.57:0/1820543624 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7fba20052020 con 0x7fba380ca600 2023-10-07T23:30:28.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 msgr2=0x7fba18062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 0x7fba18062a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba28007fd0 tx=0x7fba28007d30 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 msgr2=0x7fba3815a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fba20009f00 tx=0x7fba200091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 shutdown_connections 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba38159030 0x7fba3814f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fba380ca600 0x7fba3815a160 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fba20009f00 tx=0x7fba200091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fba18060550 0x7fba18062a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 --2- 172.21.15.57:0/1820543624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba3806cf10 0x7fba38158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.622 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 >> 172.21.15.57:0/1820543624 conn(0x7fba38068650 msgr2=0x7fba3806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:28.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 shutdown_connections 2023-10-07T23:30:28.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.616+0000 7fba3ceb4700 1 -- 172.21.15.57:0/1820543624 wait complete. 2023-10-07T23:30:28.625 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2023-10-07T23:30:28.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_metadata_threshold 2023-10-07T23:30:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.760+0000 7f5eee121700 1 Processor -- start 2023-10-07T23:30:28.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.760+0000 7f5eee121700 1 -- start start 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 0x7f5ee80ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee80c9c10 con 0x7f5ee806c800 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee806d340 con 0x7f5ee80cbe70 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee806d530 con 0x7f5ee815d0e0 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40704/0 (socket says 172.21.15.57:40704) 2023-10-07T23:30:28.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 -- 172.21.15.57:0/2195412215 learned_addr learned my addr 172.21.15.57:0/2195412215 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eed11f700 1 --2- 172.21.15.57:0/2195412215 >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 0x7f5ee80ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 -- 172.21.15.57:0/2195412215 >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 msgr2=0x7f5ee80ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 --2- 172.21.15.57:0/2195412215 >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 0x7f5ee80ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 -- 172.21.15.57:0/2195412215 >> v1:172.21.15.154:6789/0 conn(0x7f5ee806c800 legacy=0x7f5ee806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 -- 172.21.15.57:0/2195412215 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80c95c0 con 0x7f5ee80cbe70 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eed11f700 1 --2- 172.21.15.57:0/2195412215 >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 0x7f5ee80ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eec91e700 1 --2- 172.21.15.57:0/2195412215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806a240 tx=0x7f5ee4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5b9cf153ccba1d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5edbfff700 1 -- 172.21.15.57:0/2195412215 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400b860 con 0x7f5ee80cbe70 2023-10-07T23:30:28.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5edbfff700 1 -- 172.21.15.57:0/2195412215 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f5ee400b9c0 con 0x7f5ee80cbe70 2023-10-07T23:30:28.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5edbfff700 1 -- 172.21.15.57:0/2195412215 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400bd60 con 0x7f5ee80cbe70 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- 172.21.15.57:0/2195412215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee80cbe70 msgr2=0x7f5ee80cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 --2- 172.21.15.57:0/2195412215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806a240 tx=0x7f5ee4009250 comp rx=0 tx=0).stop 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- 172.21.15.57:0/2195412215 shutdown_connections 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 --2- 172.21.15.57:0/2195412215 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee80cbe70 0x7f5ee80cc2c0 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806a240 tx=0x7f5ee4009250 comp rx=0 tx=0).stop 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 --2- 172.21.15.57:0/2195412215 >> v2:172.21.15.154:3300/0 conn(0x7f5ee815d0e0 0x7f5ee80ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- 172.21.15.57:0/2195412215 >> 172.21.15.57:0/2195412215 conn(0x7f5ee8095d30 msgr2=0x7f5ee806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.764+0000 7f5eee121700 1 -- 172.21.15.57:0/2195412215 shutdown_connections 2023-10-07T23:30:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 -- 172.21.15.57:0/2195412215 wait complete. 2023-10-07T23:30:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 Processor -- start 2023-10-07T23:30:28.774 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 -- start start 2023-10-07T23:30:28.779 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee806c800 0x7f5ee815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee816d510 0x7f5ee80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80b13b0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee815e3a0 con 0x7f5ee806c800 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eee121700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee815e680 con 0x7f5ee816d510 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.768+0000 7f5eec91e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40706/0 (socket says 172.21.15.57:40706) 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 -- 172.21.15.57:0/3338855858 learned_addr learned my addr 172.21.15.57:0/3338855858 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 -- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee816d510 msgr2=0x7f5ee80c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee816d510 0x7f5ee80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 -- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee806c800 msgr2=0x7f5ee815b620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-07T23:30:28.780 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee806c800 0x7f5ee815b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 -- 172.21.15.57:0/3338855858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee40094f0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eec91e700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40090b0 tx=0x7f5ee4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401a450 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f5ee401a5d0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee401a990 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee80b1040 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f5ee401ad00 con 0x7f5ee815d0e0 2023-10-07T23:30:28.781 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee81604a0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.772+0000 7f5ed97fa700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 0x7f5ecc062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:28.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.776+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f5ee407d040 con 0x7f5ee815d0e0 2023-10-07T23:30:28.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.776+0000 7f5eed11f700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 0x7f5ecc062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:28.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.776+0000 7f5eed11f700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 0x7f5ecc062a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00094d0 tx=0x7f5ee00096b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:28.782 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.776+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee80b0ea0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.780+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ee80b0ea0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.952+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"} v 0) v1 -- 0x7f5ee806aae0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.964 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5ed97fa700 1 -- 172.21.15.57:0/3338855858 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_metadata_threshold"}]=0 v20) v1 ==== 113+0+0 (secure 0 0 0) 0x7f5ee806aae0 con 0x7f5ee815d0e0 2023-10-07T23:30:28.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 msgr2=0x7f5ecc062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 0x7f5ecc062a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00094d0 tx=0x7f5ee00096b0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 msgr2=0x7f5ee815ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:28.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40090b0 tx=0x7f5ee4009250 comp rx=0 tx=0).stop 2023-10-07T23:30:28.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 shutdown_connections 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f5ee816d510 0x7f5ee80c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ee815d0e0 0x7f5ee815ff60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f5ecc060580 0x7f5ecc062a30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 --2- 172.21.15.57:0/3338855858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ee806c800 0x7f5ee815b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 >> 172.21.15.57:0/3338855858 conn(0x7f5ee8095d30 msgr2=0x7f5ee80cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 shutdown_connections 2023-10-07T23:30:28.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:28.960+0000 7f5eee121700 1 -- 172.21.15.57:0/3338855858 wait complete. 2023-10-07T23:30:28.979 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_completed_requests 2023-10-07T23:30:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 Processor -- start 2023-10-07T23:30:29.092 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 -- start start 2023-10-07T23:30:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 0x7f3d6c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d6c0ca600 0x7f3d6c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c06a900 con 0x7f3d6c06c800 2023-10-07T23:30:29.094 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b0450 con 0x7f3d6c0ca600 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d733a2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c06a600 con 0x7f3d6c159030 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d6c0ca600 0x7f3d6c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f3d6c0ca600 0x7f3d6c0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40720/0 (socket says 172.21.15.57:40720) 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 -- 172.21.15.57:0/1024011782 learned_addr learned my addr 172.21.15.57:0/1024011782 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7113e700 1 --2- 172.21.15.57:0/1024011782 >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 0x7f3d6c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 -- 172.21.15.57:0/1024011782 >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 msgr2=0x7f3d6c0aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 --2- 172.21.15.57:0/1024011782 >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 0x7f3d6c0aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 -- 172.21.15.57:0/1024011782 >> v1:172.21.15.154:6789/0 conn(0x7f3d6c06c800 legacy=0x7f3d6c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 -- 172.21.15.57:0/1024011782 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0cb350 con 0x7f3d6c0ca600 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7113e700 1 --2- 172.21.15.57:0/1024011782 >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 0x7f3d6c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:29.095 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d7193f700 1 --2- 172.21.15.57:0/1024011782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c0caa80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b60d0 tx=0x7f3d64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec39b3da4fba911b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.096 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d63fff700 1 -- 172.21.15.57:0/1024011782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400e070 con 0x7f3d6c0ca600 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d63fff700 1 -- 172.21.15.57:0/1024011782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f3d6400ba50 con 0x7f3d6c0ca600 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.088+0000 7f3d63fff700 1 -- 172.21.15.57:0/1024011782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d640092e0 con 0x7f3d6c0ca600 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- 172.21.15.57:0/1024011782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ca600 msgr2=0x7f3d6c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 --2- 172.21.15.57:0/1024011782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c0caa80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0b60d0 tx=0x7f3d64009840 comp rx=0 tx=0).stop 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- 172.21.15.57:0/1024011782 shutdown_connections 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 --2- 172.21.15.57:0/1024011782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c0caa80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 --2- 172.21.15.57:0/1024011782 >> v2:172.21.15.154:3300/0 conn(0x7f3d6c159030 0x7f3d6c0aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- 172.21.15.57:0/1024011782 >> 172.21.15.57:0/1024011782 conn(0x7f3d6c068650 msgr2=0x7f3d6c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- 172.21.15.57:0/1024011782 shutdown_connections 2023-10-07T23:30:29.097 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- 172.21.15.57:0/1024011782 wait complete. 2023-10-07T23:30:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 Processor -- start 2023-10-07T23:30:29.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.092+0000 7f3d733a2700 1 -- start start 2023-10-07T23:30:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 0x7f3d6c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c3a30 con 0x7f3d6c159030 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b42a0 con 0x7f3d6c0ca600 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b4580 con 0x7f3d6c06c800 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 0x7f3d6c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7113e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7093d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7113e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:41108/0 (socket says 172.21.15.57:41108) 2023-10-07T23:30:29.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7113e700 1 -- 172.21.15.57:0/3584520068 learned_addr learned my addr 172.21.15.57:0/3584520068 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:29.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 -- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 msgr2=0x7f3d6c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 -- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 msgr2=0x7f3d6c14f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c14f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 -- 172.21.15.57:0/3584520068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009ae0 con 0x7f3d6c159030 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7113e700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7093d700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d7193f700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 0x7f3d6c0b1110 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3d64015000 tx=0x7f3d64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400be60 con 0x7f3d6c159030 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f3d64008d20 con 0x7f3d6c159030 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0b4860 con 0x7f3d6c159030 2023-10-07T23:30:29.104 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d640266b0 con 0x7f3d6c159030 2023-10-07T23:30:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.096+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c0b4e20 con 0x7f3d6c159030 2023-10-07T23:30:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c15a720 con 0x7f3d6c159030 2023-10-07T23:30:29.105 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3d64003040 con 0x7f3d6c159030 2023-10-07T23:30:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d617fa700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 0x7f3d50062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f3d6407e380 con 0x7f3d6c159030 2023-10-07T23:30:29.106 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d7113e700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 0x7f3d50062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.107 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.100+0000 7f3d7113e700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 0x7f3d50062a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d68006dc0 tx=0x7f3d68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.113 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.108+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3d6404ecc0 con 0x7f3d6c159030 2023-10-07T23:30:29.271 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.264+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"} v 0) v1 -- 0x7f3d6c0647b0 con 0x7f3d6c159030 2023-10-07T23:30:29.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.276+0000 7f3d617fa700 1 -- 172.21.15.57:0/3584520068 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_completed_requests"}]=0 v21) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3d640522e0 con 0x7f3d6c159030 2023-10-07T23:30:29.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 msgr2=0x7f3d50062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 0x7f3d50062a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d68006dc0 tx=0x7f3d68008040 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 msgr2=0x7f3d6c0b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 0x7f3d6c0b1110 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3d64015000 tx=0x7f3d64009840 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 shutdown_connections 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3d6c159030 0x7f3d6c0b1110 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3d50060550 0x7f3d50062a00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d6c0ca600 0x7f3d6c14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 --2- 172.21.15.57:0/3584520068 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3d6c06c800 0x7f3d6c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 >> 172.21.15.57:0/3584520068 conn(0x7f3d6c068650 msgr2=0x7f3d6c0d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:29.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 shutdown_connections 2023-10-07T23:30:29.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.280+0000 7f3d733a2700 1 -- 172.21.15.57:0/3584520068 wait complete. 2023-10-07T23:30:29.298 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm client client_inject_fixed_oldest_tid 2023-10-07T23:30:29.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 Processor -- start 2023-10-07T23:30:29.405 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 -- start start 2023-10-07T23:30:29.406 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3120159030 0x7f31200aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f312006c800 0x7f312006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312006a900 con 0x7f31200ca600 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31200b0450 con 0x7f312006c800 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f3124c0b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312006a600 con 0x7f3120159030 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f312006c800 0x7f312006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f312006c800 0x7f312006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40730/0 (socket says 172.21.15.57:40730) 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 -- 172.21.15.57:0/258039975 learned_addr learned my addr 172.21.15.57:0/258039975 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:29.407 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311e59c700 1 --2- 172.21.15.57:0/258039975 >> v2:172.21.15.154:3300/0 conn(0x7f3120159030 0x7f31200aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311d59a700 1 -- 172.21.15.57:0/258039975 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515493207 0 0) 0x7f312006a900 con 0x7f31200ca600 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311d59a700 1 -- 172.21.15.57:0/258039975 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3108003570 con 0x7f31200ca600 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 -- 172.21.15.57:0/258039975 >> v2:172.21.15.154:3300/0 conn(0x7f3120159030 msgr2=0x7f31200aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 --2- 172.21.15.57:0/258039975 >> v2:172.21.15.154:3300/0 conn(0x7f3120159030 0x7f31200aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 -- 172.21.15.57:0/258039975 >> v1:172.21.15.154:6789/0 conn(0x7f31200ca600 legacy=0x7f31200caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.400+0000 7f311dd9b700 1 -- 172.21.15.57:0/258039975 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312006d170 con 0x7f312006c800 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f311dd9b700 1 --2- 172.21.15.57:0/258039975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f312006c800 0x7f312006cc30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f31200b60d0 tx=0x7f3118009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a752604d8524bb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f311d59a700 1 -- 172.21.15.57:0/258039975 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311800e070 con 0x7f312006c800 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f311d59a700 1 -- 172.21.15.57:0/258039975 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f311800b970 con 0x7f312006c800 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f311d59a700 1 -- 172.21.15.57:0/258039975 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180092e0 con 0x7f312006c800 2023-10-07T23:30:29.409 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 -- 172.21.15.57:0/258039975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f312006c800 msgr2=0x7f312006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 --2- 172.21.15.57:0/258039975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f312006c800 0x7f312006cc30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f31200b60d0 tx=0x7f3118009840 comp rx=0 tx=0).stop 2023-10-07T23:30:29.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 -- 172.21.15.57:0/258039975 shutdown_connections 2023-10-07T23:30:29.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 --2- 172.21.15.57:0/258039975 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f312006c800 0x7f312006cc30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 --2- 172.21.15.57:0/258039975 >> v2:172.21.15.154:3300/0 conn(0x7f3120159030 0x7f31200aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.410 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 -- 172.21.15.57:0/258039975 >> 172.21.15.57:0/258039975 conn(0x7f3120068650 msgr2=0x7f3120068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:29.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 -- 172.21.15.57:0/258039975 shutdown_connections 2023-10-07T23:30:29.411 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.404+0000 7f3124c0b700 1 -- 172.21.15.57:0/258039975 wait complete. 2023-10-07T23:30:29.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 Processor -- start 2023-10-07T23:30:29.412 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- start start 2023-10-07T23:30:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f312006c800 0x7f31200b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.414 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3120159030 0x7f31200b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b47f0 con 0x7f31200ca600 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200b3cf0 con 0x7f3120159030 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31201553d0 con 0x7f312006c800 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40734/0 (socket says 172.21.15.57:40734) 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 -- 172.21.15.57:0/1441061843 learned_addr learned my addr 172.21.15.57:0/1441061843 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311e59c700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f312006c800 0x7f31200b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311ed9d700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3120159030 0x7f31200b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.415 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 -- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f312006c800 msgr2=0x7f31200b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f312006c800 0x7f31200b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 -- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3120159030 msgr2=0x7f31200b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3120159030 0x7f31200b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 -- 172.21.15.57:0/1441061843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009ae0 con 0x7f31200ca600 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f311dd9b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f31180093e0 tx=0x7f3118009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118008a40 con 0x7f31200ca600 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 46+0+0 (secure 0 0 0) 0x7f3118008bc0 con 0x7f31200ca600 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312016d3c0 con 0x7f31200ca600 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312016cb10 con 0x7f31200ca600 2023-10-07T23:30:29.416 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.408+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118026460 con 0x7f31200ca600 2023-10-07T23:30:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f3118003040 con 0x7f31200ca600 2023-10-07T23:30:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f310e7fc700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 0x7f3100062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f311807e230 con 0x7f31200ca600 2023-10-07T23:30:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f311e59c700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 0x7f3100062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.417 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31040052b0 con 0x7f31200ca600 2023-10-07T23:30:29.418 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.412+0000 7f311e59c700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 0x7f3100062a30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3110009cb0 tx=0x7f3110008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.424 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.420+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f311804eb70 con 0x7f31200ca600 2023-10-07T23:30:29.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.692+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"} v 0) v1 -- 0x7f3104005600 con 0x7f31200ca600 2023-10-07T23:30:29.706 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.700+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311804eb70 con 0x7f31200ca600 2023-10-07T23:30:29.707 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.700+0000 7f310e7fc700 1 -- 172.21.15.57:0/1441061843 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "client", "name": "client_inject_fixed_oldest_tid"}]=0 v22) v1 ==== 116+0+0 (secure 0 0 0) 0x7f3118021070 con 0x7f31200ca600 2023-10-07T23:30:29.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 msgr2=0x7f3100062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 0x7f3100062a30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3110009cb0 tx=0x7f3110008040 comp rx=0 tx=0).stop 2023-10-07T23:30:29.709 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 msgr2=0x7f312014f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f31180093e0 tx=0x7f3118009840 comp rx=0 tx=0).stop 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 shutdown_connections 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f3100060580 0x7f3100062a30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3120159030 0x7f31200b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f31200ca600 0x7f312014f370 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 --2- 172.21.15.57:0/1441061843 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f312006c800 0x7f31200b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 >> 172.21.15.57:0/1441061843 conn(0x7f3120068650 msgr2=0x7f312014fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 shutdown_connections 2023-10-07T23:30:29.710 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.704+0000 7f3124c0b700 1 -- 172.21.15.57:0/1441061843 wait complete. 2023-10-07T23:30:29.724 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-07T23:30:29.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.828+0000 7f445a5c0700 1 Processor -- start 2023-10-07T23:30:29.835 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.828+0000 7f445a5c0700 1 -- start start 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f445a5c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 0x7f44540ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f445a5c0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44540be7f0 0x7f44540bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f445a5c0700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540c9fc0 con 0x7f44540c8580 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f445a5c0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540cc820 con 0x7f44540be7f0 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f445a5c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540abe00 con 0x7f44540c8950 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f44595be700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f44540c8580 0x7f44540cb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38832/0 (socket says 172.21.15.57:38832) 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f44595be700 1 -- 172.21.15.57:0/3955824325 learned_addr learned my addr 172.21.15.57:0/3955824325 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4458dbd700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 0x7f44540ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 --2- 172.21.15.57:0/3955824325 >> v2:172.21.15.154:3300/0 conn(0x7f44540be7f0 0x7f44540bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f444bfff700 1 -- 172.21.15.57:0/3955824325 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766136346 0 0) 0x7f44540c9fc0 con 0x7f44540c8580 2023-10-07T23:30:29.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f444bfff700 1 -- 172.21.15.57:0/3955824325 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4440003570 con 0x7f44540c8580 2023-10-07T23:30:29.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 -- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 msgr2=0x7f44540ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:29.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 0x7f44540ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 -- 172.21.15.57:0/3955824325 >> v1:172.21.15.57:6789/0 conn(0x7f44540c8580 legacy=0x7f44540cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 -- 172.21.15.57:0/3955824325 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445415d850 con 0x7f44540be7f0 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4458dbd700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 0x7f44540ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f4459dbf700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540be7f0 0x7f44540bec70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44440098e0 tx=0x7f44440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199131fcb08e8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f444bfff700 1 -- 172.21.15.57:0/3955824325 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444011630 con 0x7f44540be7f0 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f444bfff700 1 -- 172.21.15.57:0/3955824325 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4444011790 con 0x7f44540be7f0 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.832+0000 7f444bfff700 1 -- 172.21.15.57:0/3955824325 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444011b10 con 0x7f44540be7f0 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540be7f0 msgr2=0x7f44540bec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540be7f0 0x7f44540bec70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44440098e0 tx=0x7f44440091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- 172.21.15.57:0/3955824325 shutdown_connections 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540be7f0 0x7f44540bec70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- 172.21.15.57:0/3955824325 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540c8950 0x7f44540ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- 172.21.15.57:0/3955824325 >> 172.21.15.57:0/3955824325 conn(0x7f44540670e0 msgr2=0x7f44540674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:29.841 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- 172.21.15.57:0/3955824325 shutdown_connections 2023-10-07T23:30:29.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- 172.21.15.57:0/3955824325 wait complete. 2023-10-07T23:30:29.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 Processor -- start 2023-10-07T23:30:29.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- start start 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540be7f0 0x7f4454160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 0x7f445415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540aacd0 con 0x7f44540c8950 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540b1210 con 0x7f44540c8580 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f445a5c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540b0600 con 0x7f44540be7f0 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.836+0000 7f4459dbf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40748/0 (socket says 172.21.15.57:40748) 2023-10-07T23:30:29.845 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 -- 172.21.15.57:0/653898045 learned_addr learned my addr 172.21.15.57:0/653898045 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44595be700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540be7f0 0x7f4454160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4458dbd700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 0x7f445415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 -- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540be7f0 msgr2=0x7f4454160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540be7f0 0x7f4454160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 -- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 msgr2=0x7f445415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 0x7f445415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 -- 172.21.15.57:0/653898045 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450009420 con 0x7f44540c8950 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4458dbd700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 0x7f445415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:29.846 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f4459dbf700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f445406c620 tx=0x7f4444017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444027450 con 0x7f44540c8950 2023-10-07T23:30:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44440275b0 con 0x7f44540c8950 2023-10-07T23:30:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4444009420 con 0x7f44540c8950 2023-10-07T23:30:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444027970 con 0x7f44540c8950 2023-10-07T23:30:29.847 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540b0990 con 0x7f44540c8950 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44540b1350 con 0x7f44540c8950 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f4444027d20 con 0x7f44540c8950 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 0x7f4438062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f444407e5b0 con 0x7f44540c8950 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.840+0000 7f44595be700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 0x7f4438062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:29.848 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.844+0000 7f44595be700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 0x7f4438062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f444c005c50 tx=0x7f444c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:29.856 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:29.852+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f444404ee70 con 0x7f44540c8950 2023-10-07T23:30:30.117 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.112+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44540711c0 con 0x7f44540c8950 2023-10-07T23:30:30.122 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.116+0000 7f44497fa700 1 -- 172.21.15.57:0/653898045 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4444052490 con 0x7f44540c8950 2023-10-07T23:30:30.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 msgr2=0x7f4438062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 0x7f4438062a00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f444c005c50 tx=0x7f444c009040 comp rx=0 tx=0).stop 2023-10-07T23:30:30.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 msgr2=0x7f445415e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.124 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f445406c620 tx=0x7f4444017000 comp rx=0 tx=0).stop 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 shutdown_connections 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44540c8950 0x7f445415e340 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f4438060550 0x7f4438062a00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44540c8580 0x7f445415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 --2- 172.21.15.57:0/653898045 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f44540be7f0 0x7f4454160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 >> 172.21.15.57:0/653898045 conn(0x7f44540670e0 msgr2=0x7f4454160eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 shutdown_connections 2023-10-07T23:30:30.125 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.120+0000 7f445a5c0700 1 -- 172.21.15.57:0/653898045 wait complete. 2023-10-07T23:30:30.136 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2023-10-07T23:30:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.236+0000 7f50c0ee2700 1 Processor -- start 2023-10-07T23:30:30.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.236+0000 7f50c0ee2700 1 -- start start 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.236+0000 7f50c0ee2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158be0 0x7f50bc0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.236+0000 7f50c0ee2700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50bc158fb0 0x7f50bc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc06b3c0 con 0x7f50bc06b850 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc06a7b0 con 0x7f50bc158fb0 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc06aa90 con 0x7f50bc158be0 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f50bc158fb0 0x7f50bc0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50bbfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f50bc06b850 0x7f50bc0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38838/0 (socket says 172.21.15.57:38838) 2023-10-07T23:30:30.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50bbfff700 1 -- 172.21.15.57:0/4107165958 learned_addr learned my addr 172.21.15.57:0/4107165958 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50bb7fe700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158be0 0x7f50bc0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50ba7fc700 1 -- 172.21.15.57:0/4107165958 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273108706 0 0) 0x7f50bc06b3c0 con 0x7f50bc06b850 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50ba7fc700 1 -- 172.21.15.57:0/4107165958 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a4003570 con 0x7f50bc06b850 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 -- 172.21.15.57:0/4107165958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158be0 msgr2=0x7f50bc0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158be0 0x7f50bc0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 -- 172.21.15.57:0/4107165958 >> v1:172.21.15.57:6789/0 conn(0x7f50bc06b850 legacy=0x7f50bc0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 -- 172.21.15.57:0/4107165958 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc06ad70 con 0x7f50bc158fb0 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50baffd700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158fb0 0x7f50bc0ce6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50b00098e0 tx=0x7f50b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62017b07168e21d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50ba7fc700 1 -- 172.21.15.57:0/4107165958 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0011630 con 0x7f50bc158fb0 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50ba7fc700 1 -- 172.21.15.57:0/4107165958 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0011790 con 0x7f50bc158fb0 2023-10-07T23:30:30.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50ba7fc700 1 -- 172.21.15.57:0/4107165958 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0011b30 con 0x7f50bc158fb0 2023-10-07T23:30:30.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- 172.21.15.57:0/4107165958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158fb0 msgr2=0x7f50bc0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158fb0 0x7f50bc0ce6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50b00098e0 tx=0x7f50b00091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- 172.21.15.57:0/4107165958 shutdown_connections 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158fb0 0x7f50bc0ce6b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/4107165958 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158be0 0x7f50bc0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- 172.21.15.57:0/4107165958 >> 172.21.15.57:0/4107165958 conn(0x7f50bc068520 msgr2=0x7f50bc068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- 172.21.15.57:0/4107165958 shutdown_connections 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- 172.21.15.57:0/4107165958 wait complete. 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 Processor -- start 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.240+0000 7f50c0ee2700 1 -- start start 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc06b850 0x7f50bc15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158fb0 0x7f50bc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc155840 con 0x7f50bc158be0 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0c9910 con 0x7f50bc06b850 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0c9bf0 con 0x7f50bc158fb0 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40768/0 (socket says 172.21.15.57:40768) 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 -- 172.21.15.57:0/1484778289 learned_addr learned my addr 172.21.15.57:0/1484778289 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50bbfff700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158fb0 0x7f50bc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50bb7fe700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc06b850 0x7f50bc15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 -- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158fb0 msgr2=0x7f50bc0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158fb0 0x7f50bc0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 -- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc06b850 msgr2=0x7f50bc15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc06b850 0x7f50bc15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.250 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 -- 172.21.15.57:0/1484778289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b0009420 con 0x7f50bc158be0 2023-10-07T23:30:30.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50baffd700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009ed0 tx=0x7f50b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0011900 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b0011a60 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0025600 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc15e3e0 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc155a90 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc0c9d80 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.244+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f50b00259f0 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.248+0000 7f50a37fe700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 0x7f50a8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.248+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f50b007db40 con 0x7f50bc158be0 2023-10-07T23:30:30.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.248+0000 7f50bb7fe700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 0x7f50a8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:30.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.248+0000 7f50bb7fe700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 0x7f50a8062a30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009db0 tx=0x7f50ac0091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:30.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.252+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f50b004e400 con 0x7f50bc158be0 2023-10-07T23:30:30.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.464+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) v1 -- 0x7f50bc0711c0 con 0x7f50bc158be0 2023-10-07T23:30:30.624 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.620+0000 7f50a37fe700 1 -- 172.21.15.57:0/1484778289 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v342) v1 ==== 162+0+0 (secure 0 0 0) 0x7f50b0015020 con 0x7f50bc158be0 2023-10-07T23:30:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 msgr2=0x7f50a8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 0x7f50a8062a30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009db0 tx=0x7f50ac0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 msgr2=0x7f50bc15dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:30.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009ed0 tx=0x7f50b0017000 comp rx=0 tx=0).stop 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 shutdown_connections 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f50bc158fb0 0x7f50bc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f50bc158be0 0x7f50bc15dea0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f50a8060580 0x7f50a8062a30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 --2- 172.21.15.57:0/1484778289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc06b850 0x7f50bc15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 >> 172.21.15.57:0/1484778289 conn(0x7f50bc068520 msgr2=0x7f50bc0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 shutdown_connections 2023-10-07T23:30:30.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:30.624+0000 7f50c0ee2700 1 -- 172.21.15.57:0/1484778289 wait complete. 2023-10-07T23:30:30.645 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... ERROR 2023-10-07T23:30:30.645 INFO:tasks.cephfs_test_runner: 2023-10-07T23:30:30.645 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:ERROR: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_client_limits.py", line 282, in test_client_blocklisted_oldest_tid 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner: self.assertGreater(pd['mds_sessions']['md_thresh_evicted'], 0) 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:KeyError: 'md_thresh_evicted' 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner: 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:Ran 2 tests in 517.165s 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner: 2023-10-07T23:30:30.646 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner: 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:ERROR: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_client_limits.py", line 282, in test_client_blocklisted_oldest_tid 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner: self.assertGreater(pd['mds_sessions']['md_thresh_evicted'], 0) 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner:KeyError: 'md_thresh_evicted' 2023-10-07T23:30:30.647 INFO:tasks.cephfs_test_runner: 2023-10-07T23:30:30.648 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:30:30.714 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=794d5b6462a14cc2b83f29ceac4a104f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2023-10-07T23:30:30.716 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-07T23:30:30.726 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-07T23:30:30.772 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-07T23:30:30.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:30.773 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2023-10-07T23:30:30.794 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-07T23:30:30.794 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-07T23:30:30.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:30.795 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-07T23:30:30.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:30.809 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-07T23:30:30.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:30.870 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2023-10-07T23:30:30.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:30.997 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-10-07T23:30:31.007 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-07T23:30:31.008 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-07T23:30:31.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:31.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-07T23:30:31.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:31.020 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-07T23:30:31.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-07T23:30:31.036 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-10-07T23:30:31.192 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-07T23:30:31.201 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-07T23:30:31.202 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:31.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.308+0000 7fd22929d700 1 Processor -- start 2023-10-07T23:30:31.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 -- start start 2023-10-07T23:30:31.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd224158be0 0x7fd2240ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd224158fb0 0x7fd2240ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22406b3c0 con 0x7fd22406b850 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22406a7b0 con 0x7fd224158be0 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd22929d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22406aa90 con 0x7fd224158fb0 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fd224158be0 0x7fd2240ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd228a9c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fd22406b850 0x7fd2240cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38844/0 (socket says 172.21.15.57:38844) 2023-10-07T23:30:31.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd228a9c700 1 -- 172.21.15.57:0/3254889896 learned_addr learned my addr 172.21.15.57:0/3254889896 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:31.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd21bfff700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd224158fb0 0x7fd2240ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd2237fe700 1 -- 172.21.15.57:0/3254889896 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394097617 0 0) 0x7fd22406b3c0 con 0x7fd22406b850 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd2237fe700 1 -- 172.21.15.57:0/3254889896 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003570 con 0x7fd22406b850 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 -- 172.21.15.57:0/3254889896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd224158fb0 msgr2=0x7fd2240ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd224158fb0 0x7fd2240ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 -- 172.21.15.57:0/3254889896 >> v1:172.21.15.57:6789/0 conn(0x7fd22406b850 legacy=0x7fd2240cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 -- 172.21.15.57:0/3254889896 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22406ad70 con 0x7fd224158be0 2023-10-07T23:30:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd223fff700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2240ce170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd2140098e0 tx=0x7fd2140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87d21765c4f20c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd2237fe700 1 -- 172.21.15.57:0/3254889896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214011630 con 0x7fd224158be0 2023-10-07T23:30:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd2237fe700 1 -- 172.21.15.57:0/3254889896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214011790 con 0x7fd224158be0 2023-10-07T23:30:31.321 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.312+0000 7fd2237fe700 1 -- 172.21.15.57:0/3254889896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214011b10 con 0x7fd224158be0 2023-10-07T23:30:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 -- 172.21.15.57:0/3254889896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 msgr2=0x7fd2240ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2240ce170 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd2140098e0 tx=0x7fd2140091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 -- 172.21.15.57:0/3254889896 shutdown_connections 2023-10-07T23:30:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd224158fb0 0x7fd2240ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.322 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 --2- 172.21.15.57:0/3254889896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2240ce170 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 -- 172.21.15.57:0/3254889896 >> 172.21.15.57:0/3254889896 conn(0x7fd224068520 msgr2=0x7fd224068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:31.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 -- 172.21.15.57:0/3254889896 shutdown_connections 2023-10-07T23:30:31.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 -- 172.21.15.57:0/3254889896 wait complete. 2023-10-07T23:30:31.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.316+0000 7fd22929d700 1 Processor -- start 2023-10-07T23:30:31.324 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- start start 2023-10-07T23:30:31.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd22406b850 0x7fd22414c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 0x7fd2241508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22414f860 con 0x7fd224158be0 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2241544f0 con 0x7fd224158fb0 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22414ef30 con 0x7fd22406b850 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40774/0 (socket says 172.21.15.57:40774) 2023-10-07T23:30:31.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 -- 172.21.15.57:0/2716093619 learned_addr learned my addr 172.21.15.57:0/2716093619 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd223fff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd22406b850 0x7fd22414c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd228a9c700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 0x7fd2241508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 -- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd22406b850 msgr2=0x7fd22414c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd22406b850 0x7fd22414c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 -- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 msgr2=0x7fd2241508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 0x7fd2241508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 -- 172.21.15.57:0/2716093619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009420 con 0x7fd224158be0 2023-10-07T23:30:31.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd228a9c700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 0x7fd2241508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd21bfff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ec80 tx=0x7fd20c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00c850 con 0x7fd224158be0 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c00c9b0 con 0x7fd224158be0 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22414f210 con 0x7fd224158be0 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00cd70 con 0x7fd224158be0 2023-10-07T23:30:31.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.320+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22414c950 con 0x7fd224158be0 2023-10-07T23:30:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22414f9d0 con 0x7fd224158be0 2023-10-07T23:30:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fd20c013680 con 0x7fd224158be0 2023-10-07T23:30:31.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd220ff9700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 0x7fd204062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fd20c07c6f0 con 0x7fd224158be0 2023-10-07T23:30:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd223fff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 0x7fd204062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.324+0000 7fd223fff700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 0x7fd204062a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2140091e0 tx=0x7fd214017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.332+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd20c04cfb0 con 0x7fd224158be0 2023-10-07T23:30:31.496 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.492+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2240711c0 con 0x7fd224158be0 2023-10-07T23:30:31.497 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.492+0000 7fd220ff9700 1 -- 172.21.15.57:0/2716093619 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12324 (secure 0 0 0) 0x7fd20c0505d0 con 0x7fd224158be0 2023-10-07T23:30:31.498 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:31.498 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":53,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:28.596220+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/320776638":"2023-10-08T23:30:25.586325+0000","172.21.15.154:6834/320776638":"2023-10-08T23:30:25.586325+0000","192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:31.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 msgr2=0x7fd204062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.500 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 0x7fd204062a30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2140091e0 tx=0x7fd214017000 comp rx=0 tx=0).stop 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 msgr2=0x7fd2241503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00ec80 tx=0x7fd20c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 shutdown_connections 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fd204060580 0x7fd204062a30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd224158fb0 0x7fd2241508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd224158be0 0x7fd2241503a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.501 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 --2- 172.21.15.57:0/2716093619 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fd22406b850 0x7fd22414c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 >> 172.21.15.57:0/2716093619 conn(0x7fd224068520 msgr2=0x7fd22414d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:31.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 shutdown_connections 2023-10-07T23:30:31.502 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.496+0000 7fd22929d700 1 -- 172.21.15.57:0/2716093619 wait complete. 2023-10-07T23:30:31.513 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-07T23:30:31.513 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:30:31.623 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.616+0000 7f15ad6b5700 1 Processor -- start 2023-10-07T23:30:31.625 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.616+0000 7f15ad6b5700 1 -- start start 2023-10-07T23:30:31.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a8158d60 0x7f15a806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a806bac0 0x7f15a80c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a806bec0 con 0x7f15a8159130 2023-10-07T23:30:31.628 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80c81d0 con 0x7f15a8158d60 2023-10-07T23:30:31.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80c83b0 con 0x7f15a806bac0 2023-10-07T23:30:31.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a8158d60 0x7f15a806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f159ffff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f15a8159130 0x7f15a80c7580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38852/0 (socket says 172.21.15.57:38852) 2023-10-07T23:30:31.629 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f159ffff700 1 -- 172.21.15.57:0/3988438842 learned_addr learned my addr 172.21.15.57:0/3988438842 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:31.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f15a8158d60 0x7f15a806b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40790/0 (socket says 172.21.15.57:40790) 2023-10-07T23:30:31.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15aceb4700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a806bac0 0x7f15a80c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a77fe700 1 -- 172.21.15.57:0/3988438842 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933690489 0 0) 0x7f15a806bec0 con 0x7f15a8159130 2023-10-07T23:30:31.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a77fe700 1 -- 172.21.15.57:0/3988438842 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1588003570 con 0x7f15a8159130 2023-10-07T23:30:31.630 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 -- 172.21.15.57:0/3988438842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a806bac0 msgr2=0x7f15a80c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a806bac0 0x7f15a80c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 -- 172.21.15.57:0/3988438842 >> v1:172.21.15.57:6789/0 conn(0x7f15a8159130 legacy=0x7f15a80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 -- 172.21.15.57:0/3988438842 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80c8690 con 0x7f15a8158d60 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a7fff700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8158d60 0x7f15a806b580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f15980098e0 tx=0x7f15980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d300428a203a573 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a77fe700 1 -- 172.21.15.57:0/3988438842 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011630 con 0x7f15a8158d60 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a77fe700 1 -- 172.21.15.57:0/3988438842 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598011790 con 0x7f15a8158d60 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15a77fe700 1 -- 172.21.15.57:0/3988438842 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011b10 con 0x7f15a8158d60 2023-10-07T23:30:31.631 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- 172.21.15.57:0/3988438842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8158d60 msgr2=0x7f15a806b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8158d60 0x7f15a806b580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f15980098e0 tx=0x7f15980091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- 172.21.15.57:0/3988438842 shutdown_connections 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a806bac0 0x7f15a80c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/3988438842 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8158d60 0x7f15a806b580 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- 172.21.15.57:0/3988438842 >> 172.21.15.57:0/3988438842 conn(0x7f15a80670e0 msgr2=0x7f15a80674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- 172.21.15.57:0/3988438842 shutdown_connections 2023-10-07T23:30:31.632 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.620+0000 7f15ad6b5700 1 -- 172.21.15.57:0/3988438842 wait complete. 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 Processor -- start 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 -- start start 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 0x7f15a80ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8158d60 0x7f15a8169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.635 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80c45d0 con 0x7f15a8159130 2023-10-07T23:30:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a816a310 con 0x7f15a806bac0 2023-10-07T23:30:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15ad6b5700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80ca5e0 con 0x7f15a8158d60 2023-10-07T23:30:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.636 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40792/0 (socket says 172.21.15.57:40792) 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 -- 172.21.15.57:0/2523100160 learned_addr learned my addr 172.21.15.57:0/2523100160 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f159ffff700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8158d60 0x7f15a8169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15a7fff700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 0x7f15a80ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 -- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8158d60 msgr2=0x7f15a8169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8158d60 0x7f15a8169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 -- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 msgr2=0x7f15a80ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.637 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 0x7f15a80ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15aceb4700 1 -- 172.21.15.57:0/2523100160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009420 con 0x7f15a8159130 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.624+0000 7f15a7fff700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 0x7f15a80ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15aceb4700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009e90 tx=0x7f15a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cbc0 con 0x7f15a8159130 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000f040 con 0x7f15a8159130 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a80ca8c0 con 0x7f15a8159130 2023-10-07T23:30:31.638 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0014480 con 0x7f15a8159130 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80cac50 con 0x7f15a8159130 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a80711c0 con 0x7f15a8159130 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f15a00147f0 con 0x7f15a8159130 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 0x7f158c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f15a007d140 con 0x7f15a8159130 2023-10-07T23:30:31.639 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.628+0000 7f15a7fff700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 0x7f158c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.640 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.632+0000 7f15a7fff700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 0x7f158c062a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15980098e0 tx=0x7f1598017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.641 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.636+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f15a0051490 con 0x7f15a8159130 2023-10-07T23:30:31.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.792+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15a80c8970 con 0x7f158c060550 2023-10-07T23:30:31.802 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.796+0000 7f15a4ff9700 1 -- 172.21.15.57:0/2523100160 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49028 (secure 0 0 0) 0x7f15a80c8970 con 0x7f158c060550 2023-10-07T23:30:31.803 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 msgr2=0x7f158c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.804 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 0x7f158c062a00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f15980098e0 tx=0x7f1598017000 comp rx=0 tx=0).stop 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 msgr2=0x7f15a80b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009e90 tx=0x7f15a000b040 comp rx=0 tx=0).stop 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 shutdown_connections 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f15a8159130 0x7f15a80b0ce0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8158d60 0x7f15a8169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f158c060550 0x7f158c062a00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.805 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 --2- 172.21.15.57:0/2523100160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a806bac0 0x7f15a80ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 >> 172.21.15.57:0/2523100160 conn(0x7f15a80670e0 msgr2=0x7f15a80674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:31.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 shutdown_connections 2023-10-07T23:30:31.806 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.800+0000 7f15ad6b5700 1 -- 172.21.15.57:0/2523100160 wait complete. 2023-10-07T23:30:31.806 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:30:31.818 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-10-07T23:30:31.258298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2996916,"kb_used_data":565364,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":751977804,"statfs":{"total":773094113280,"available":770025271296,"internally_reserved":0,"allocated":578932736,"data_stored":325362814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-182108136,"num_objects":-34744,"num_object_clones":0,"num_object_copies":-69488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34744,"num_whiteouts":0,"num_read":-342,"num_read_kb":0,"num_write":-63160,"num_write_kb":-229738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-168,"num_bytes_recovered":-645,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001964"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":186,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-07T23:30:20.040190+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:30:20.040190+0000","last_peered":"2023-10-07T23:30:20.040190+0000","last_clean":"2023-10-07T23:30:20.040190+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:30:20.040190+0000","last_undegraded":"2023-10-07T23:30:20.040190+0000","last_fullsized":"2023-10-07T23:30:20.040190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":387648,"kb_used_data":78072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93984192,"statfs":{"total":96636764160,"available":96239812608,"internally_reserved":0,"allocated":79945728,"data_stored":48602786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":0.89400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0920000000000001,"5min":2.48,"15min":2.48},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"min":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.53900000000000003},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3240000000000001,"5min":1.409,"15min":1.4510000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.41899999999999998},"max":{"1min":1.286,"5min":1.397,"15min":1.5069999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.98699999999999999},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0269999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3460000000000001,"5min":1.494,"15min":1.7310000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.02,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.628,"15min":0.51100000000000001},"max":{"1min":1.7310000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0169999999999999,"15min":1.0109999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":2.7839999999999998},"last":1.095}]},{"osd":4,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4390000000000001,"5min":1.569,"15min":1.603},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0580000000000001,"15min":1.0860000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.554,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.052}]},{"osd":5,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.79},"last":1.222},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1519999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.905},"last":1.2}]},{"osd":6,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7549999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.645,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.1519999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384432,"kb_used_data":83048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93987408,"statfs":{"total":96636764160,"available":96243105792,"internally_reserved":0,"allocated":85041152,"data_stored":38040651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.88500000000000001},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.038,"5min":1.575,"15min":1.764},"last":1.181},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89800000000000002,"15min":0.95499999999999996},"min":{"1min":0.751,"5min":0.57899999999999996,"15min":0.54500000000000004},"max":{"1min":1.236,"5min":1.2989999999999999,"15min":1.6379999999999999},"last":1.296}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78200000000000003,"15min":0.88200000000000001},"min":{"1min":0.63500000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.05,"5min":1.2190000000000001,"15min":1.5389999999999999},"last":1.012},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.339,"15min":1.726},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.65800000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.26,"5min":1.351,"15min":1.4299999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.88700000000000001,"15min":0.93700000000000006},"min":{"1min":0.81200000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.161,"5min":1.3460000000000001,"15min":1.484},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.002,"15min":1.0269999999999999},"min":{"1min":0.77800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4079999999999999,"5min":1.5600000000000001,"15min":1.6970000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.117,"5min":1.0640000000000001,"15min":1.1140000000000001},"min":{"1min":0.92200000000000004,"5min":0.65800000000000003,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.5409999999999999,"15min":2.145},"last":0.70199999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91200000000000003,"15min":0.95899999999999996},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.492,"5min":1.6830000000000001,"15min":1.7529999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94899999999999995,"15min":1.016},"min":{"1min":0.60799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.327,"5min":1.4319999999999999,"15min":2.2749999999999999},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.042},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3140000000000001,"5min":1.617,"15min":1.617},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0649999999999999,"15min":1.129},"min":{"1min":0.91100000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3939999999999999,"5min":1.7629999999999999,"15min":1.8280000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.88400000000000001,"5min":0.63,"15min":0.53600000000000003},"max":{"1min":1.4370000000000001,"5min":1.675,"15min":2.0640000000000001},"last":1.206},{"interface":"front","average":{"1min":1.196,"5min":1.1020000000000001,"15min":1.149},"min":{"1min":0.97699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5700000000000001,"15min":1.671},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395972,"kb_used_data":86396,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93975868,"statfs":{"total":96636764160,"available":96231288832,"internally_reserved":0,"allocated":88469504,"data_stored":58992967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51500000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"max":{"1min":1.109,"5min":1.919,"15min":1.919},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90900000000000003,"15min":0.99099999999999999},"min":{"1min":0.54700000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2649999999999999,"5min":1.458,"15min":2.4950000000000001},"last":1.252}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80900000000000005,"15min":0.90500000000000003},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.1180000000000001,"5min":1.306,"15min":2.3450000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.86599999999999999},"min":{"1min":0.69899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3129999999999999,"5min":1.653,"15min":1.653},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.47899999999999998},"max":{"1min":1.2609999999999999,"5min":1.3280000000000001,"15min":2.1549999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":1.0600000000000001},"min":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":1.373,"15min":2.452},"last":1.2250000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94799999999999995,"15min":0.98899999999999999},"min":{"1min":0.66600000000000004,"5min":0.57199999999999995,"15min":0.51700000000000002},"max":{"1min":1.367,"5min":1.5449999999999999,"15min":2.649},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96199999999999997,"15min":1.0289999999999999},"min":{"1min":0.82799999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.175,"5min":1.3540000000000001,"15min":2.077},"last":1.2749999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.66300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3799999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.96899999999999997,"15min":1.056},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3160000000000001,"5min":1.538,"15min":2.395},"last":1.1830000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.98699999999999999,"15min":1.0960000000000001},"min":{"1min":0.76700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4319999999999999,"15min":3.1840000000000002},"last":1.155},{"interface":"front","average":{"1min":1.125,"5min":1.0569999999999999,"15min":1.087},"min":{"1min":0.77800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.454,"5min":1.764,"15min":1.764},"last":1.1319999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4370000000000001,"5min":1.623,"15min":2.7229999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.107,"15min":1.2250000000000001},"min":{"1min":0.97499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.585,"5min":1.671,"15min":3.3029999999999999},"last":1.3240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366724,"kb_used_data":69436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94005116,"statfs":{"total":96636764160,"available":96261238784,"internally_reserved":0,"allocated":71102464,"data_stored":42215589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76300000000000001,"15min":0.84399999999999997},"min":{"1min":0.51200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.671},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":0.97199999999999998},"min":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.518},"last":1.28}]},{"osd":1,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.0469999999999999,"5min":1.2729999999999999,"15min":1.5649999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.86299999999999999,"15min":0.88900000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.206,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.88100000000000001,"15min":0.93799999999999994},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.248,"5min":1.4430000000000001,"15min":1.952},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0860000000000001,"15min":1.143},"min":{"1min":0.751,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":2.0750000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.94699999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.47799999999999998},"max":{"1min":1.4630000000000001,"5min":1.6299999999999999,"15min":2.3330000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.022,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.327,"5min":1.597,"15min":1.7749999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.41399999999999998},"max":{"1min":1.579,"5min":1.744,"15min":1.744},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.997,"15min":1.0589999999999999},"min":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.659,"5min":1.661,"15min":1.9630000000000001},"last":0.875}]},{"osd":6,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92400000000000004,"15min":0.98199999999999998},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1970000000000001,"5min":1.617,"15min":1.706},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.038,"15min":1.1100000000000001},"min":{"1min":0.70699999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":1.792,"15min":2.3900000000000001},"last":1.032}]},{"osd":7,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1040000000000001,"15min":1.1439999999999999},"min":{"1min":0.79700000000000004,"5min":0.54000000000000004,"15min":0.39000000000000001},"max":{"1min":1.782,"5min":1.8360000000000001,"15min":1.9379999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.1220000000000001},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.60899999999999999},"max":{"1min":1.522,"5min":1.6930000000000001,"15min":1.919},"last":1.089}]}]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403852,"kb_used_data":98372,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93967988,"statfs":{"total":96636764160,"available":96223219712,"internally_reserved":0,"allocated":100732928,"data_stored":64795104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.1759999999999999,"15min":1.2769999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.83599999999999997},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.33200000000000002},"max":{"1min":1.242,"5min":1.274,"15min":2.8769999999999998},"last":0.64900000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.71399999999999997},"min":{"1min":0.53200000000000003,"5min":0.44800000000000001,"15min":0.35599999999999998},"max":{"1min":1.1100000000000001,"5min":1.23,"15min":1.2529999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91800000000000004},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.34899999999999998},"max":{"1min":1.2450000000000001,"5min":1.4450000000000001,"15min":2.5369999999999999},"last":1.103}]},{"osd":2,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85999999999999999,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.375},"max":{"1min":1.161,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.74299999999999999,"5min":0.64600000000000002,"15min":0.502},"max":{"1min":1.3340000000000001,"5min":1.665,"15min":2.7389999999999999},"last":1.1839999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.034,"15min":1.0860000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.64200000000000002},"max":{"1min":1.339,"5min":1.6180000000000001,"15min":2.044},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.036,"15min":1.0129999999999999},"min":{"1min":0.746,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.3300000000000001,"5min":1.704,"15min":1.704},"last":0.85299999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.0620000000000001},"min":{"1min":0.84599999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.476,"5min":1.585,"15min":1.649},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.173,"15min":1.228},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4790000000000001,"5min":1.728,"15min":2.6709999999999998},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2050000000000001,"15min":1.2729999999999999},"min":{"1min":0.91700000000000004,"5min":0.73699999999999999,"15min":0.69999999999999996},"max":{"1min":1.5720000000000001,"5min":1.804,"15min":2.802},"last":0.998},{"interface":"front","average":{"1min":1.208,"5min":1.2270000000000001,"15min":1.2889999999999999},"min":{"1min":0.81799999999999995,"5min":0.72599999999999998,"15min":0.64000000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.758},"last":1.0680000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2689999999999999,"15min":1.278},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.625,"5min":1.7709999999999999,"15min":2.2090000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.21,"15min":1.1970000000000001},"min":{"1min":0.92500000000000004,"5min":0.84899999999999998,"15min":0.65900000000000003},"max":{"1min":1.446,"5min":1.827,"15min":2.601},"last":0.93100000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350944,"kb_used_data":53656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94020896,"statfs":{"total":96636764160,"available":96277397504,"internally_reserved":0,"allocated":54943744,"data_stored":21249284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":2.1739999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.34899999999999998},"max":{"1min":1.1259999999999999,"5min":1.381,"15min":1.827},"last":0.75900000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":2.2690000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.32500000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.9019999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89700000000000002,"15min":0.85499999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0860000000000001,"15min":0.97999999999999998},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.44700000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.2229999999999999},"last":1.3}]},{"osd":4,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.071,"15min":1.0600000000000001},"min":{"1min":0.75800000000000001,"5min":0.71299999999999997,"15min":0.56699999999999995},"max":{"1min":1.587,"5min":1.853,"15min":2.0699999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.0960000000000001},"min":{"1min":0.68700000000000006,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.88800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.087,"15min":1.1080000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.484,"5min":1.657,"15min":2.302},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2150000000000001,"15min":1.1699999999999999},"min":{"1min":0.75,"5min":0.68200000000000005,"15min":0.64000000000000001},"max":{"1min":1.534,"5min":1.829,"15min":2.1819999999999999},"last":1.143}]},{"osd":6,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.56000000000000005},"max":{"1min":1.6619999999999999,"5min":1.9199999999999999,"15min":2.2389999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1799999999999999,"15min":1.2130000000000001},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":2.4460000000000002},"last":1.1120000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.276,"15min":1.2689999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6579999999999999,"5min":1.8660000000000001,"15min":2.282},"last":1.079},{"interface":"front","average":{"1min":1.129,"5min":1.2030000000000001,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"max":{"1min":1.427,"5min":1.6359999999999999,"15min":10.183},"last":1.173}]}]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":397156,"kb_used_data":95772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93974684,"statfs":{"total":96636764160,"available":96230076416,"internally_reserved":0,"allocated":98070528,"data_stored":51213159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.93200000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.02}]},{"osd":2,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.86599999999999999,"15min":0.80900000000000005},"min":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.86499999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.78200000000000003},"min":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.36799999999999999},"max":{"1min":0.93500000000000005,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.026,"15min":0.97099999999999997},"min":{"1min":0.65600000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.71199999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.077,"15min":1.095},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.427,"5min":1.7689999999999999,"15min":2.3879999999999999},"last":1.139},{"interface":"front","average":{"1min":1.02,"5min":1.0249999999999999,"15min":1.0620000000000001},"min":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.47999999999999998},"max":{"1min":1.2669999999999999,"5min":1.8799999999999999,"15min":2.1920000000000002},"last":1.2230000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.63800000000000001},"max":{"1min":1.752,"5min":2.278,"15min":2.278},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.2090000000000001,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5349999999999999,"5min":1.9399999999999999,"15min":2.2759999999999998},"last":1.4079999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.141,"15min":1.1719999999999999},"min":{"1min":0.79600000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":2.0910000000000002,"15min":2.7189999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.264,"15min":1.2589999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.639,"5min":2.375,"15min":2.375},"last":1.181}]},{"osd":7,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3320000000000001,"15min":1.298},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.70199999999999996},"max":{"1min":1.6299999999999999,"5min":2.5030000000000001,"15min":2.524},"last":1.266},{"interface":"front","average":{"1min":1.155,"5min":1.2270000000000001,"15min":1.238},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.413,"5min":2.1030000000000002,"15min":2.3330000000000002},"last":1.1040000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.2130000000000001,"15min":2.5129999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.33900000000000002},"max":{"1min":1.093,"5min":1.1419999999999999,"15min":1.1890000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94199999999999995,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.452,"15min":1.6439999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.92600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.42299999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.5089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.0189999999999999,"15min":0.95699999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.42899999999999999},"max":{"1min":1.3440000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":1.798,"15min":1.798},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.046},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.478,"5min":1.603,"15min":1.603},"last":0.999}]},{"osd":5,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"max":{"1min":1.371,"5min":1.548,"15min":1.548},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.1719999999999999,"15min":1.1659999999999999},"min":{"1min":0.84299999999999997,"5min":0.745,"15min":0.53900000000000003},"max":{"1min":1.397,"5min":1.6859999999999999,"15min":1.8129999999999999},"last":0.876}]},{"osd":6,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.202},"min":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.57299999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.895},"last":1.071},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.22,"15min":1.2130000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.67200000000000004},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.802},"last":0.89900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.248,"15min":1.248},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.64500000000000002},"max":{"1min":1.3540000000000001,"5min":1.7809999999999999,"15min":1.8939999999999999},"last":1.022},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.27,"15min":1.27},"min":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.625},"max":{"1min":1.6930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:30:31.820 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:30:31.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.924+0000 7f8923fff700 1 Processor -- start 2023-10-07T23:30:31.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.924+0000 7f8923fff700 1 -- start start 2023-10-07T23:30:31.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8924158c40 0x7f8924159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89240ca530 con 0x7f892415d580 2023-10-07T23:30:31.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892406ae60 con 0x7f892406a6f0 2023-10-07T23:30:31.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892406b140 con 0x7f8924158c40 2023-10-07T23:30:31.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f892415d580 0x7f89240afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38868/0 (socket says 172.21.15.57:38868) 2023-10-07T23:30:31.933 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 -- 172.21.15.57:0/2534589372 learned_addr learned my addr 172.21.15.57:0/2534589372 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:31.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8922ffd700 1 --2- 172.21.15.57:0/2534589372 >> v2:172.21.15.154:3300/0 conn(0x7f8924158c40 0x7f8924159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759677588 0 0) 0x7f89240ca530 con 0x7f892415d580 2023-10-07T23:30:31.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f890c003570 con 0x7f892415d580 2023-10-07T23:30:31.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2319169300 0 0) 0x7f890c003570 con 0x7f892415d580 2023-10-07T23:30:31.938 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 >> v2:172.21.15.154:3300/0 conn(0x7f8924158c40 msgr2=0x7f8924159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:31.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 --2- 172.21.15.57:0/2534589372 >> v2:172.21.15.154:3300/0 conn(0x7f8924158c40 0x7f8924159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.939 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 >> v1:172.21.15.154:6789/0 conn(0x7f892406a6f0 legacy=0x7f892406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892406b420 con 0x7f892415d580 2023-10-07T23:30:31.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7f8914001470 con 0x7f892415d580 2023-10-07T23:30:31.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 >> v1:172.21.15.57:6789/0 conn(0x7f892415d580 legacy=0x7f89240afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c003a60 0x7f890c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.940 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f890c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f890c0051c0 0x7f890c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892406b140 con 0x7f890c004810 2023-10-07T23:30:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c00c0d0 con 0x7f890c003a60 2023-10-07T23:30:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c00c290 con 0x7f890c0051c0 2023-10-07T23:30:31.941 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f890c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8922ffd700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c003a60 0x7f890c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89237fe700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f890c0051c0 0x7f890c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 -- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f890c0051c0 msgr2=0x7f890c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:31.942 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f890c0051c0 0x7f890c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 -- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c003a60 msgr2=0x7f890c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f890c003a60 0x7f890c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 -- 172.21.15.57:0/2534589372 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c003570 con 0x7f890c004810 2023-10-07T23:30:31.943 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89237fe700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f890c0051c0 0x7f890c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:31.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89237fe700 1 -- 172.21.15.57:0/2534589372 reap_dead start 2023-10-07T23:30:31.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f89227fc700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f890c004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f891400c390 tx=0x7f89240ca4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a81de29be3b29fd server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400cc60 con 0x7f890c004810 2023-10-07T23:30:31.944 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891400cdc0 con 0x7f890c004810 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8921ffb700 1 -- 172.21.15.57:0/2534589372 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914005b70 con 0x7f890c004810 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 msgr2=0x7f890c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f890c004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f891400c390 tx=0x7f89240ca4a0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- 172.21.15.57:0/2534589372 shutdown_connections 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 --2- 172.21.15.57:0/2534589372 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f890c004c80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.945 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- 172.21.15.57:0/2534589372 >> 172.21.15.57:0/2534589372 conn(0x7f8924062ba0 msgr2=0x7f8924062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- 172.21.15.57:0/2534589372 shutdown_connections 2023-10-07T23:30:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.928+0000 7f8923fff700 1 -- 172.21.15.57:0/2534589372 wait complete. 2023-10-07T23:30:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.932+0000 7f8923fff700 1 Processor -- start 2023-10-07T23:30:31.946 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.932+0000 7f8923fff700 1 -- start start 2023-10-07T23:30:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.932+0000 7f8923fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.932+0000 7f8923fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924159fa0 0x7f892415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f892415ec40 0x7f892415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.949 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892415f5f0 con 0x7f890c004810 2023-10-07T23:30:31.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892415acc0 con 0x7f8924159fa0 2023-10-07T23:30:31.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892415af10 con 0x7f892415ec40 2023-10-07T23:30:31.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40804/0 (socket says 172.21.15.57:40804) 2023-10-07T23:30:31.950 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 -- 172.21.15.57:0/1659853836 learned_addr learned my addr 172.21.15.57:0/1659853836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f89227fc700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924159fa0 0x7f892415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f89237fe700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f892415ec40 0x7f892415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 -- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f892415ec40 msgr2=0x7f892415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f892415ec40 0x7f892415f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 -- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924159fa0 msgr2=0x7f892415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924159fa0 0x7f892415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 -- 172.21.15.57:0/1659853836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892415b1f0 con 0x7f890c004810 2023-10-07T23:30:31.951 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8922ffd700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8918011d40 tx=0x7f89180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891800fb50 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891800fcb0 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892415b4d0 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918014440 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892415ba90 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89240711c0 con 0x7f890c004810 2023-10-07T23:30:31.952 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.936+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8918014a80 con 0x7f890c004810 2023-10-07T23:30:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.940+0000 7f890affd700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 0x7f8910062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.940+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f8918018030 con 0x7f890c004810 2023-10-07T23:30:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.940+0000 7f89227fc700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 0x7f8910062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.940+0000 7f89227fc700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 0x7f8910062a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8914007cd0 tx=0x7f8914015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:31.953 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:31.944+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f891804dbd0 con 0x7f890c004810 2023-10-07T23:30:32.110 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.104+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89240af620 con 0x7f8910060580 2023-10-07T23:30:32.116 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f890affd700 1 -- 172.21.15.57:0/1659853836 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49028 (secure 0 0 0) 0x7f89240af620 con 0x7f8910060580 2023-10-07T23:30:32.117 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:32.118 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 msgr2=0x7f8910062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 0x7f8910062a30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8914007cd0 tx=0x7f8914015000 comp rx=0 tx=0).stop 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 msgr2=0x7f8924157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8918011d40 tx=0x7f89180091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 shutdown_connections 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f892415ec40 0x7f892415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8910060580 0x7f8910062a30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.119 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8924159fa0 0x7f892415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 --2- 172.21.15.57:0/1659853836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f890c004810 0x7f8924157650 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 >> 172.21.15.57:0/1659853836 conn(0x7f8924062ba0 msgr2=0x7f89240aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:32.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 shutdown_connections 2023-10-07T23:30:32.120 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.112+0000 7f8923fff700 1 -- 172.21.15.57:0/1659853836 wait complete. 2023-10-07T23:30:32.120 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:30:32.132 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-10-07T23:30:31.258298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2996916,"kb_used_data":565364,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":751977804,"statfs":{"total":773094113280,"available":770025271296,"internally_reserved":0,"allocated":578932736,"data_stored":325362814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-182108136,"num_objects":-34744,"num_object_clones":0,"num_object_copies":-69488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34744,"num_whiteouts":0,"num_read":-342,"num_read_kb":0,"num_write":-63160,"num_write_kb":-229738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-168,"num_bytes_recovered":-645,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001964"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":186,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-07T23:30:20.040190+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:30:20.040190+0000","last_peered":"2023-10-07T23:30:20.040190+0000","last_clean":"2023-10-07T23:30:20.040190+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:30:20.040190+0000","last_undegraded":"2023-10-07T23:30:20.040190+0000","last_fullsized":"2023-10-07T23:30:20.040190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":387648,"kb_used_data":78072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93984192,"statfs":{"total":96636764160,"available":96239812608,"internally_reserved":0,"allocated":79945728,"data_stored":48602786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":0.89400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0920000000000001,"5min":2.48,"15min":2.48},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"min":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.53900000000000003},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3240000000000001,"5min":1.409,"15min":1.4510000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.41899999999999998},"max":{"1min":1.286,"5min":1.397,"15min":1.5069999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.98699999999999999},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0269999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3460000000000001,"5min":1.494,"15min":1.7310000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.02,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.628,"15min":0.51100000000000001},"max":{"1min":1.7310000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0169999999999999,"15min":1.0109999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":2.7839999999999998},"last":1.095}]},{"osd":4,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4390000000000001,"5min":1.569,"15min":1.603},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0580000000000001,"15min":1.0860000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.554,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.052}]},{"osd":5,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.79},"last":1.222},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1519999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.905},"last":1.2}]},{"osd":6,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7549999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.645,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.1519999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384432,"kb_used_data":83048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93987408,"statfs":{"total":96636764160,"available":96243105792,"internally_reserved":0,"allocated":85041152,"data_stored":38040651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.88500000000000001},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.038,"5min":1.575,"15min":1.764},"last":1.181},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89800000000000002,"15min":0.95499999999999996},"min":{"1min":0.751,"5min":0.57899999999999996,"15min":0.54500000000000004},"max":{"1min":1.236,"5min":1.2989999999999999,"15min":1.6379999999999999},"last":1.296}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78200000000000003,"15min":0.88200000000000001},"min":{"1min":0.63500000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.05,"5min":1.2190000000000001,"15min":1.5389999999999999},"last":1.012},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.339,"15min":1.726},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.65800000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.26,"5min":1.351,"15min":1.4299999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.88700000000000001,"15min":0.93700000000000006},"min":{"1min":0.81200000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.161,"5min":1.3460000000000001,"15min":1.484},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.002,"15min":1.0269999999999999},"min":{"1min":0.77800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4079999999999999,"5min":1.5600000000000001,"15min":1.6970000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.117,"5min":1.0640000000000001,"15min":1.1140000000000001},"min":{"1min":0.92200000000000004,"5min":0.65800000000000003,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.5409999999999999,"15min":2.145},"last":0.70199999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91200000000000003,"15min":0.95899999999999996},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.492,"5min":1.6830000000000001,"15min":1.7529999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94899999999999995,"15min":1.016},"min":{"1min":0.60799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.327,"5min":1.4319999999999999,"15min":2.2749999999999999},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.042},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3140000000000001,"5min":1.617,"15min":1.617},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0649999999999999,"15min":1.129},"min":{"1min":0.91100000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3939999999999999,"5min":1.7629999999999999,"15min":1.8280000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.88400000000000001,"5min":0.63,"15min":0.53600000000000003},"max":{"1min":1.4370000000000001,"5min":1.675,"15min":2.0640000000000001},"last":1.206},{"interface":"front","average":{"1min":1.196,"5min":1.1020000000000001,"15min":1.149},"min":{"1min":0.97699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5700000000000001,"15min":1.671},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395972,"kb_used_data":86396,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93975868,"statfs":{"total":96636764160,"available":96231288832,"internally_reserved":0,"allocated":88469504,"data_stored":58992967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51500000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"max":{"1min":1.109,"5min":1.919,"15min":1.919},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90900000000000003,"15min":0.99099999999999999},"min":{"1min":0.54700000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2649999999999999,"5min":1.458,"15min":2.4950000000000001},"last":1.252}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80900000000000005,"15min":0.90500000000000003},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.1180000000000001,"5min":1.306,"15min":2.3450000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.86599999999999999},"min":{"1min":0.69899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3129999999999999,"5min":1.653,"15min":1.653},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.47899999999999998},"max":{"1min":1.2609999999999999,"5min":1.3280000000000001,"15min":2.1549999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":1.0600000000000001},"min":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":1.373,"15min":2.452},"last":1.2250000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94799999999999995,"15min":0.98899999999999999},"min":{"1min":0.66600000000000004,"5min":0.57199999999999995,"15min":0.51700000000000002},"max":{"1min":1.367,"5min":1.5449999999999999,"15min":2.649},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96199999999999997,"15min":1.0289999999999999},"min":{"1min":0.82799999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.175,"5min":1.3540000000000001,"15min":2.077},"last":1.2749999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.66300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3799999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.96899999999999997,"15min":1.056},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3160000000000001,"5min":1.538,"15min":2.395},"last":1.1830000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.98699999999999999,"15min":1.0960000000000001},"min":{"1min":0.76700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4319999999999999,"15min":3.1840000000000002},"last":1.155},{"interface":"front","average":{"1min":1.125,"5min":1.0569999999999999,"15min":1.087},"min":{"1min":0.77800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.454,"5min":1.764,"15min":1.764},"last":1.1319999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4370000000000001,"5min":1.623,"15min":2.7229999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.107,"15min":1.2250000000000001},"min":{"1min":0.97499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.585,"5min":1.671,"15min":3.3029999999999999},"last":1.3240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366724,"kb_used_data":69436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94005116,"statfs":{"total":96636764160,"available":96261238784,"internally_reserved":0,"allocated":71102464,"data_stored":42215589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76300000000000001,"15min":0.84399999999999997},"min":{"1min":0.51200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.671},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":0.97199999999999998},"min":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.518},"last":1.28}]},{"osd":1,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.0469999999999999,"5min":1.2729999999999999,"15min":1.5649999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.86299999999999999,"15min":0.88900000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.206,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.88100000000000001,"15min":0.93799999999999994},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.248,"5min":1.4430000000000001,"15min":1.952},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0860000000000001,"15min":1.143},"min":{"1min":0.751,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":2.0750000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.94699999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.47799999999999998},"max":{"1min":1.4630000000000001,"5min":1.6299999999999999,"15min":2.3330000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.022,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.327,"5min":1.597,"15min":1.7749999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.41399999999999998},"max":{"1min":1.579,"5min":1.744,"15min":1.744},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.997,"15min":1.0589999999999999},"min":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.659,"5min":1.661,"15min":1.9630000000000001},"last":0.875}]},{"osd":6,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92400000000000004,"15min":0.98199999999999998},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1970000000000001,"5min":1.617,"15min":1.706},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.038,"15min":1.1100000000000001},"min":{"1min":0.70699999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":1.792,"15min":2.3900000000000001},"last":1.032}]},{"osd":7,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1040000000000001,"15min":1.1439999999999999},"min":{"1min":0.79700000000000004,"5min":0.54000000000000004,"15min":0.39000000000000001},"max":{"1min":1.782,"5min":1.8360000000000001,"15min":1.9379999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.1220000000000001},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.60899999999999999},"max":{"1min":1.522,"5min":1.6930000000000001,"15min":1.919},"last":1.089}]}]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403852,"kb_used_data":98372,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93967988,"statfs":{"total":96636764160,"available":96223219712,"internally_reserved":0,"allocated":100732928,"data_stored":64795104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.1759999999999999,"15min":1.2769999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.83599999999999997},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.33200000000000002},"max":{"1min":1.242,"5min":1.274,"15min":2.8769999999999998},"last":0.64900000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.71399999999999997},"min":{"1min":0.53200000000000003,"5min":0.44800000000000001,"15min":0.35599999999999998},"max":{"1min":1.1100000000000001,"5min":1.23,"15min":1.2529999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91800000000000004},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.34899999999999998},"max":{"1min":1.2450000000000001,"5min":1.4450000000000001,"15min":2.5369999999999999},"last":1.103}]},{"osd":2,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85999999999999999,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.375},"max":{"1min":1.161,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.74299999999999999,"5min":0.64600000000000002,"15min":0.502},"max":{"1min":1.3340000000000001,"5min":1.665,"15min":2.7389999999999999},"last":1.1839999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.034,"15min":1.0860000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.64200000000000002},"max":{"1min":1.339,"5min":1.6180000000000001,"15min":2.044},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.036,"15min":1.0129999999999999},"min":{"1min":0.746,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.3300000000000001,"5min":1.704,"15min":1.704},"last":0.85299999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.0620000000000001},"min":{"1min":0.84599999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.476,"5min":1.585,"15min":1.649},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.173,"15min":1.228},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4790000000000001,"5min":1.728,"15min":2.6709999999999998},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2050000000000001,"15min":1.2729999999999999},"min":{"1min":0.91700000000000004,"5min":0.73699999999999999,"15min":0.69999999999999996},"max":{"1min":1.5720000000000001,"5min":1.804,"15min":2.802},"last":0.998},{"interface":"front","average":{"1min":1.208,"5min":1.2270000000000001,"15min":1.2889999999999999},"min":{"1min":0.81799999999999995,"5min":0.72599999999999998,"15min":0.64000000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.758},"last":1.0680000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2689999999999999,"15min":1.278},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.625,"5min":1.7709999999999999,"15min":2.2090000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.21,"15min":1.1970000000000001},"min":{"1min":0.92500000000000004,"5min":0.84899999999999998,"15min":0.65900000000000003},"max":{"1min":1.446,"5min":1.827,"15min":2.601},"last":0.93100000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350944,"kb_used_data":53656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94020896,"statfs":{"total":96636764160,"available":96277397504,"internally_reserved":0,"allocated":54943744,"data_stored":21249284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":2.1739999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.34899999999999998},"max":{"1min":1.1259999999999999,"5min":1.381,"15min":1.827},"last":0.75900000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":2.2690000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.32500000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.9019999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89700000000000002,"15min":0.85499999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0860000000000001,"15min":0.97999999999999998},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.44700000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.2229999999999999},"last":1.3}]},{"osd":4,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.071,"15min":1.0600000000000001},"min":{"1min":0.75800000000000001,"5min":0.71299999999999997,"15min":0.56699999999999995},"max":{"1min":1.587,"5min":1.853,"15min":2.0699999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.0960000000000001},"min":{"1min":0.68700000000000006,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.88800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.087,"15min":1.1080000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.484,"5min":1.657,"15min":2.302},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2150000000000001,"15min":1.1699999999999999},"min":{"1min":0.75,"5min":0.68200000000000005,"15min":0.64000000000000001},"max":{"1min":1.534,"5min":1.829,"15min":2.1819999999999999},"last":1.143}]},{"osd":6,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.56000000000000005},"max":{"1min":1.6619999999999999,"5min":1.9199999999999999,"15min":2.2389999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1799999999999999,"15min":1.2130000000000001},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":2.4460000000000002},"last":1.1120000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.276,"15min":1.2689999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6579999999999999,"5min":1.8660000000000001,"15min":2.282},"last":1.079},{"interface":"front","average":{"1min":1.129,"5min":1.2030000000000001,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"max":{"1min":1.427,"5min":1.6359999999999999,"15min":10.183},"last":1.173}]}]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":397156,"kb_used_data":95772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93974684,"statfs":{"total":96636764160,"available":96230076416,"internally_reserved":0,"allocated":98070528,"data_stored":51213159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.93200000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.02}]},{"osd":2,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.86599999999999999,"15min":0.80900000000000005},"min":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.86499999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.78200000000000003},"min":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.36799999999999999},"max":{"1min":0.93500000000000005,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.026,"15min":0.97099999999999997},"min":{"1min":0.65600000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.71199999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.077,"15min":1.095},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.427,"5min":1.7689999999999999,"15min":2.3879999999999999},"last":1.139},{"interface":"front","average":{"1min":1.02,"5min":1.0249999999999999,"15min":1.0620000000000001},"min":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.47999999999999998},"max":{"1min":1.2669999999999999,"5min":1.8799999999999999,"15min":2.1920000000000002},"last":1.2230000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.63800000000000001},"max":{"1min":1.752,"5min":2.278,"15min":2.278},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.2090000000000001,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5349999999999999,"5min":1.9399999999999999,"15min":2.2759999999999998},"last":1.4079999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.141,"15min":1.1719999999999999},"min":{"1min":0.79600000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":2.0910000000000002,"15min":2.7189999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.264,"15min":1.2589999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.639,"5min":2.375,"15min":2.375},"last":1.181}]},{"osd":7,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3320000000000001,"15min":1.298},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.70199999999999996},"max":{"1min":1.6299999999999999,"5min":2.5030000000000001,"15min":2.524},"last":1.266},{"interface":"front","average":{"1min":1.155,"5min":1.2270000000000001,"15min":1.238},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.413,"5min":2.1030000000000002,"15min":2.3330000000000002},"last":1.1040000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.2130000000000001,"15min":2.5129999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.33900000000000002},"max":{"1min":1.093,"5min":1.1419999999999999,"15min":1.1890000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94199999999999995,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.452,"15min":1.6439999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.92600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.42299999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.5089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.0189999999999999,"15min":0.95699999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.42899999999999999},"max":{"1min":1.3440000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":1.798,"15min":1.798},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.046},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.478,"5min":1.603,"15min":1.603},"last":0.999}]},{"osd":5,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"max":{"1min":1.371,"5min":1.548,"15min":1.548},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.1719999999999999,"15min":1.1659999999999999},"min":{"1min":0.84299999999999997,"5min":0.745,"15min":0.53900000000000003},"max":{"1min":1.397,"5min":1.6859999999999999,"15min":1.8129999999999999},"last":0.876}]},{"osd":6,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.202},"min":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.57299999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.895},"last":1.071},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.22,"15min":1.2130000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.67200000000000004},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.802},"last":0.89900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.248,"15min":1.248},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.64500000000000002},"max":{"1min":1.3540000000000001,"5min":1.7809999999999999,"15min":1.8939999999999999},"last":1.022},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.27,"15min":1.27},"min":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.625},"max":{"1min":1.6930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:30:32.134 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-07T23:30:32.134 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:30:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 Processor -- start 2023-10-07T23:30:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 -- start start 2023-10-07T23:30:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe0240cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0240cac40 con 0x7fe0240bee60 2023-10-07T23:30:32.244 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0240cc3c0 con 0x7fe0240c6c10 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02b6c3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240c7780 con 0x7fe0240cba50 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02945f700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fe0240bee60 0x7fe0240cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38872/0 (socket says 172.21.15.57:38872) 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02945f700 1 -- 172.21.15.57:0/658367826 learned_addr learned my addr 172.21.15.57:0/658367826 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe028c5e700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe0240cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811174010 0 0) 0x7fe0240cac40 con 0x7fe0240bee60 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe010003570 con 0x7fe0240bee60 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975414892 0 0) 0x7fe0240cc3c0 con 0x7fe0240c6c10 2023-10-07T23:30:32.245 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0240cac40 con 0x7fe0240c6c10 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441055864 0 0) 0x7fe010003570 con 0x7fe0240bee60 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 msgr2=0x7fe0240cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe0240cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 >> v1:172.21.15.154:6789/0 conn(0x7fe0240c6c10 legacy=0x7fe0240c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240ca030 con 0x7fe0240bee60 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7fe018001470 con 0x7fe0240bee60 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 >> v1:172.21.15.57:6789/0 conn(0x7fe0240bee60 legacy=0x7fe0240cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.246 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe010003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe010004180 0x7fe010004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe010004d50 0x7fe0100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240c7780 con 0x7fe010004d50 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000bba0 con 0x7fe010004180 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000bd60 con 0x7fe0240cba50 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe010004d50 0x7fe0100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe028c5e700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe010004180 0x7fe010004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.247 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe02945f700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe010003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 -- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 msgr2=0x7fe010003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240cba50 0x7fe010003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 -- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe010004180 msgr2=0x7fe010004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe010004180 0x7fe010004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe029c60700 1 -- 172.21.15.57:0/658367826 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010003570 con 0x7fe010004d50 2023-10-07T23:30:32.248 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.236+0000 7fe028c5e700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe010004180 0x7fe010004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:32.249 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe029c60700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe010004d50 0x7fe0100051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe020001a10 tx=0x7fe020007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7a4e1111505415 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.251 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000c5d0 con 0x7fe010004d50 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02000c730 con 0x7fe010004d50 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe017fff700 1 -- 172.21.15.57:0/658367826 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000cad0 con 0x7fe010004d50 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- 172.21.15.57:0/658367826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe010004d50 msgr2=0x7fe0100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/658367826 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe010004d50 0x7fe0100051c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe020001a10 tx=0x7fe020007d30 comp rx=0 tx=0).stop 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe029c60700 1 -- 172.21.15.57:0/658367826 reap_dead start 2023-10-07T23:30:32.252 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- 172.21.15.57:0/658367826 shutdown_connections 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- 172.21.15.57:0/658367826 >> 172.21.15.57:0/658367826 conn(0x7fe0240670e0 msgr2=0x7fe0240674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- 172.21.15.57:0/658367826 shutdown_connections 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- 172.21.15.57:0/658367826 wait complete. 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 Processor -- start 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.240+0000 7fe02b6c3700 1 -- start start 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe024150df0 0x7fe0240c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.253 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240aa630 0x7fe0240aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02416db40 con 0x7fe0241511c0 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240aafe0 con 0x7fe024150df0 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02416dd20 con 0x7fe0240aa630 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40820/0 (socket says 172.21.15.57:40820) 2023-10-07T23:30:32.254 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 -- 172.21.15.57:0/1522778648 learned_addr learned my addr 172.21.15.57:0/1522778648 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02945f700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe024150df0 0x7fe0240c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe029c60700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240aa630 0x7fe0240aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 -- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240aa630 msgr2=0x7fe0240aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240aa630 0x7fe0240aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 -- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe024150df0 msgr2=0x7fe0240c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe024150df0 0x7fe0240c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.255 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 -- 172.21.15.57:0/1522778648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02416e000 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe028c5e700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00f560 tx=0x7fe01c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00e040 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00f040 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c015680 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02416e2e0 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02416e8a0 con 0x7fe0241511c0 2023-10-07T23:30:32.256 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.244+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0240711c0 con 0x7fe0241511c0 2023-10-07T23:30:32.257 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.248+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fe01c003020 con 0x7fe0241511c0 2023-10-07T23:30:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.248+0000 7fe0157fa700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 0x7fe008062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.248+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fe01c07cf40 con 0x7fe0241511c0 2023-10-07T23:30:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.248+0000 7fe02945f700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 0x7fe008062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.258 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.248+0000 7fe02945f700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 0x7fe008062a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe024160c60 tx=0x7fe018001df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.256+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe01c04d800 con 0x7fe0241511c0 2023-10-07T23:30:32.423 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.416+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0240ca310 con 0x7fe008060550 2023-10-07T23:30:32.428 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe0157fa700 1 -- 172.21.15.57:0/1522778648 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49028 (secure 0 0 0) 0x7fe0240ca310 con 0x7fe008060550 2023-10-07T23:30:32.429 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:32.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 msgr2=0x7fe008062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 0x7fe008062a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe024160c60 tx=0x7fe018001df0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 msgr2=0x7fe0240c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00f560 tx=0x7fe01c00b040 comp rx=0 tx=0).stop 2023-10-07T23:30:32.431 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 shutdown_connections 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe0240aa630 0x7fe0240aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe0241511c0 0x7fe0240c4b00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fe008060550 0x7fe008062a00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 --2- 172.21.15.57:0/1522778648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe024150df0 0x7fe0240c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 >> 172.21.15.57:0/1522778648 conn(0x7fe0240670e0 msgr2=0x7fe02415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 shutdown_connections 2023-10-07T23:30:32.432 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.424+0000 7fe02b6c3700 1 -- 172.21.15.57:0/1522778648 wait complete. 2023-10-07T23:30:32.433 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:30:32.444 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-10-07T23:30:31.258298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2996916,"kb_used_data":565364,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":751977804,"statfs":{"total":773094113280,"available":770025271296,"internally_reserved":0,"allocated":578932736,"data_stored":325362814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-182108136,"num_objects":-34744,"num_object_clones":0,"num_object_copies":-69488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34744,"num_whiteouts":0,"num_read":-342,"num_read_kb":0,"num_write":-63160,"num_write_kb":-229738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-168,"num_bytes_recovered":-645,"num_keys_recovered":0,"num_objects_omap":-22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001964"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":186,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-07T23:30:20.040190+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:30:20.040190+0000","last_peered":"2023-10-07T23:30:20.040190+0000","last_clean":"2023-10-07T23:30:20.040190+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:30:20.040190+0000","last_undegraded":"2023-10-07T23:30:20.040190+0000","last_fullsized":"2023-10-07T23:30:20.040190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":387648,"kb_used_data":78072,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93984192,"statfs":{"total":96636764160,"available":96239812608,"internally_reserved":0,"allocated":79945728,"data_stored":48602786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":0.89400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0920000000000001,"5min":2.48,"15min":2.48},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"min":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.53900000000000003},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3240000000000001,"5min":1.409,"15min":1.4510000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.41899999999999998},"max":{"1min":1.286,"5min":1.397,"15min":1.5069999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.98699999999999999},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0269999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3460000000000001,"5min":1.494,"15min":1.7310000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.02,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.628,"15min":0.51100000000000001},"max":{"1min":1.7310000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0169999999999999,"15min":1.0109999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":2.7839999999999998},"last":1.095}]},{"osd":4,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4390000000000001,"5min":1.569,"15min":1.603},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0580000000000001,"15min":1.0860000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.554,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.052}]},{"osd":5,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.79},"last":1.222},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1519999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.905},"last":1.2}]},{"osd":6,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7549999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.645,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.1519999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384432,"kb_used_data":83048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93987408,"statfs":{"total":96636764160,"available":96243105792,"internally_reserved":0,"allocated":85041152,"data_stored":38040651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.88500000000000001},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.038,"5min":1.575,"15min":1.764},"last":1.181},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89800000000000002,"15min":0.95499999999999996},"min":{"1min":0.751,"5min":0.57899999999999996,"15min":0.54500000000000004},"max":{"1min":1.236,"5min":1.2989999999999999,"15min":1.6379999999999999},"last":1.296}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78200000000000003,"15min":0.88200000000000001},"min":{"1min":0.63500000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.05,"5min":1.2190000000000001,"15min":1.5389999999999999},"last":1.012},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.86399999999999999,"15min":0.90700000000000003},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":1.1060000000000001,"5min":1.339,"15min":1.726},"last":0.97199999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.65800000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.26,"5min":1.351,"15min":1.4299999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.88700000000000001,"15min":0.93700000000000006},"min":{"1min":0.81200000000000006,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.161,"5min":1.3460000000000001,"15min":1.484},"last":1.1539999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.002,"15min":1.0269999999999999},"min":{"1min":0.77800000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4079999999999999,"5min":1.5600000000000001,"15min":1.6970000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.117,"5min":1.0640000000000001,"15min":1.1140000000000001},"min":{"1min":0.92200000000000004,"5min":0.65800000000000003,"15min":0.60899999999999999},"max":{"1min":1.3580000000000001,"5min":1.5409999999999999,"15min":2.145},"last":0.70199999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91200000000000003,"15min":0.95899999999999996},"min":{"1min":0.63200000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.492,"5min":1.6830000000000001,"15min":1.7529999999999999},"last":1.083},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.94899999999999995,"15min":1.016},"min":{"1min":0.60799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.327,"5min":1.4319999999999999,"15min":2.2749999999999999},"last":1.0629999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.042},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3140000000000001,"5min":1.617,"15min":1.617},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0649999999999999,"15min":1.129},"min":{"1min":0.91100000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3939999999999999,"5min":1.7629999999999999,"15min":1.8280000000000001},"last":1.2509999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.0920000000000001,"15min":1.141},"min":{"1min":0.88400000000000001,"5min":0.63,"15min":0.53600000000000003},"max":{"1min":1.4370000000000001,"5min":1.675,"15min":2.0640000000000001},"last":1.206},{"interface":"front","average":{"1min":1.196,"5min":1.1020000000000001,"15min":1.149},"min":{"1min":0.97699999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5680000000000001,"5min":1.5700000000000001,"15min":1.671},"last":1.2330000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395972,"kb_used_data":86396,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":93975868,"statfs":{"total":96636764160,"available":96231288832,"internally_reserved":0,"allocated":88469504,"data_stored":58992967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.51500000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"max":{"1min":1.109,"5min":1.919,"15min":1.919},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90900000000000003,"15min":0.99099999999999999},"min":{"1min":0.54700000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2649999999999999,"5min":1.458,"15min":2.4950000000000001},"last":1.252}]},{"osd":1,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.80900000000000005,"15min":0.90500000000000003},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.1180000000000001,"5min":1.306,"15min":2.3450000000000002},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.85099999999999998,"15min":0.86599999999999999},"min":{"1min":0.69899999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3129999999999999,"5min":1.653,"15min":1.653},"last":0.74099999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.47899999999999998},"max":{"1min":1.2609999999999999,"5min":1.3280000000000001,"15min":2.1549999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.98199999999999998,"15min":1.0600000000000001},"min":{"1min":0.69199999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2110000000000001,"5min":1.373,"15min":2.452},"last":1.2250000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.94799999999999995,"15min":0.98899999999999999},"min":{"1min":0.66600000000000004,"5min":0.57199999999999995,"15min":0.51700000000000002},"max":{"1min":1.367,"5min":1.5449999999999999,"15min":2.649},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96199999999999997,"15min":1.0289999999999999},"min":{"1min":0.82799999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.175,"5min":1.3540000000000001,"15min":2.077},"last":1.2749999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.66300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3799999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.96899999999999997,"15min":1.056},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3160000000000001,"5min":1.538,"15min":2.395},"last":1.1830000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":0.98699999999999999,"15min":1.0960000000000001},"min":{"1min":0.76700000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.4139999999999999,"5min":1.4319999999999999,"15min":3.1840000000000002},"last":1.155},{"interface":"front","average":{"1min":1.125,"5min":1.0569999999999999,"15min":1.087},"min":{"1min":0.77800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.454,"5min":1.764,"15min":1.764},"last":1.1319999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:29:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.107,"15min":1.1639999999999999},"min":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4370000000000001,"5min":1.623,"15min":2.7229999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.107,"15min":1.2250000000000001},"min":{"1min":0.97499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.585,"5min":1.671,"15min":3.3029999999999999},"last":1.3240000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366724,"kb_used_data":69436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94005116,"statfs":{"total":96636764160,"available":96261238784,"internally_reserved":0,"allocated":71102464,"data_stored":42215589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76300000000000001,"15min":0.84399999999999997},"min":{"1min":0.51200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.671},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.91700000000000004,"15min":0.97199999999999998},"min":{"1min":0.59299999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.363,"5min":1.363,"15min":1.518},"last":1.28}]},{"osd":1,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.0469999999999999,"5min":1.2729999999999999,"15min":1.5649999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.86299999999999999,"15min":0.88900000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.206,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.88100000000000001,"15min":0.93799999999999994},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.248,"5min":1.4430000000000001,"15min":1.952},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0860000000000001,"15min":1.143},"min":{"1min":0.751,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":2.0750000000000002},"last":1.1160000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.91200000000000003,"15min":0.94699999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.47799999999999998},"max":{"1min":1.4630000000000001,"5min":1.6299999999999999,"15min":2.3330000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.022,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.327,"5min":1.597,"15min":1.7749999999999999},"last":1.2030000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.52200000000000002,"5min":0.47699999999999998,"15min":0.41399999999999998},"max":{"1min":1.579,"5min":1.744,"15min":1.744},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.997,"15min":1.0589999999999999},"min":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.659,"5min":1.661,"15min":1.9630000000000001},"last":0.875}]},{"osd":6,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92400000000000004,"15min":0.98199999999999998},"min":{"1min":0.57199999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1970000000000001,"5min":1.617,"15min":1.706},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.038,"15min":1.1100000000000001},"min":{"1min":0.70699999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":1.792,"15min":2.3900000000000001},"last":1.032}]},{"osd":7,"last update":"Sat Oct 7 23:29:32 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1040000000000001,"15min":1.1439999999999999},"min":{"1min":0.79700000000000004,"5min":0.54000000000000004,"15min":0.39000000000000001},"max":{"1min":1.782,"5min":1.8360000000000001,"15min":1.9379999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.1220000000000001},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.60899999999999999},"max":{"1min":1.522,"5min":1.6930000000000001,"15min":1.919},"last":1.089}]}]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403852,"kb_used_data":98372,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":93967988,"statfs":{"total":96636764160,"available":96223219712,"internally_reserved":0,"allocated":100732928,"data_stored":64795104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66100000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.1759999999999999,"15min":1.2769999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.83599999999999997},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.33200000000000002},"max":{"1min":1.242,"5min":1.274,"15min":2.8769999999999998},"last":0.64900000000000002}]},{"osd":1,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.71399999999999997},"min":{"1min":0.53200000000000003,"5min":0.44800000000000001,"15min":0.35599999999999998},"max":{"1min":1.1100000000000001,"5min":1.23,"15min":1.2529999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91800000000000004},"min":{"1min":0.57199999999999995,"5min":0.52300000000000002,"15min":0.34899999999999998},"max":{"1min":1.2450000000000001,"5min":1.4450000000000001,"15min":2.5369999999999999},"last":1.103}]},{"osd":2,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85999999999999999,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.375},"max":{"1min":1.161,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.74299999999999999,"5min":0.64600000000000002,"15min":0.502},"max":{"1min":1.3340000000000001,"5min":1.665,"15min":2.7389999999999999},"last":1.1839999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.034,"15min":1.0860000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.64200000000000002},"max":{"1min":1.339,"5min":1.6180000000000001,"15min":2.044},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.004,"5min":1.036,"15min":1.0129999999999999},"min":{"1min":0.746,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.3300000000000001,"5min":1.704,"15min":1.704},"last":0.85299999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1040000000000001,"15min":1.0620000000000001},"min":{"1min":0.84599999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.476,"5min":1.585,"15min":1.649},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.173,"15min":1.228},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4790000000000001,"5min":1.728,"15min":2.6709999999999998},"last":0.89700000000000002}]},{"osd":6,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2050000000000001,"15min":1.2729999999999999},"min":{"1min":0.91700000000000004,"5min":0.73699999999999999,"15min":0.69999999999999996},"max":{"1min":1.5720000000000001,"5min":1.804,"15min":2.802},"last":0.998},{"interface":"front","average":{"1min":1.208,"5min":1.2270000000000001,"15min":1.2889999999999999},"min":{"1min":0.81799999999999995,"5min":0.72599999999999998,"15min":0.64000000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.758},"last":1.0680000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:29:31 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2689999999999999,"15min":1.278},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.625,"5min":1.7709999999999999,"15min":2.2090000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.21,"15min":1.1970000000000001},"min":{"1min":0.92500000000000004,"5min":0.84899999999999998,"15min":0.65900000000000003},"max":{"1min":1.446,"5min":1.827,"15min":2.601},"last":0.93100000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":350944,"kb_used_data":53656,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94020896,"statfs":{"total":96636764160,"available":96277397504,"internally_reserved":0,"allocated":54943744,"data_stored":21249284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":2.1739999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.34899999999999998},"max":{"1min":1.1259999999999999,"5min":1.381,"15min":1.827},"last":0.75900000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":2.2690000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.32500000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.9019999999999999},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89700000000000002,"15min":0.85499999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0860000000000001,"15min":0.97999999999999998},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.44700000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.2229999999999999},"last":1.3}]},{"osd":4,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.071,"15min":1.0600000000000001},"min":{"1min":0.75800000000000001,"5min":0.71299999999999997,"15min":0.56699999999999995},"max":{"1min":1.587,"5min":1.853,"15min":2.0699999999999998},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.0960000000000001},"min":{"1min":0.68700000000000006,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.88800000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.087,"15min":1.1080000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.484,"5min":1.657,"15min":2.302},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2150000000000001,"15min":1.1699999999999999},"min":{"1min":0.75,"5min":0.68200000000000005,"15min":0.64000000000000001},"max":{"1min":1.534,"5min":1.829,"15min":2.1819999999999999},"last":1.143}]},{"osd":6,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.56000000000000005},"max":{"1min":1.6619999999999999,"5min":1.9199999999999999,"15min":2.2389999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1799999999999999,"15min":1.2130000000000001},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":2.4460000000000002},"last":1.1120000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.276,"15min":1.2689999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6579999999999999,"5min":1.8660000000000001,"15min":2.282},"last":1.079},{"interface":"front","average":{"1min":1.129,"5min":1.2030000000000001,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"max":{"1min":1.427,"5min":1.6359999999999999,"15min":10.183},"last":1.173}]}]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":397156,"kb_used_data":95772,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":93974684,"statfs":{"total":96636764160,"available":96230076416,"internally_reserved":0,"allocated":98070528,"data_stored":51213159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.41399999999999998},"max":{"1min":0.93200000000000005,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":1.02}]},{"osd":2,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.83999999999999997,"15min":0.81599999999999995},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.86599999999999999,"15min":0.80900000000000005},"min":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.50900000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.86499999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.78200000000000003},"min":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.36799999999999999},"max":{"1min":0.93500000000000005,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.026,"15min":0.97099999999999997},"min":{"1min":0.65600000000000003,"5min":0.51000000000000001,"15min":0.50600000000000001},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.71199999999999997}]},{"osd":4,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.077,"15min":1.095},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.427,"5min":1.7689999999999999,"15min":2.3879999999999999},"last":1.139},{"interface":"front","average":{"1min":1.02,"5min":1.0249999999999999,"15min":1.0620000000000001},"min":{"1min":0.78300000000000003,"5min":0.67100000000000004,"15min":0.47999999999999998},"max":{"1min":1.2669999999999999,"5min":1.8799999999999999,"15min":2.1920000000000002},"last":1.2230000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.2,"15min":1.1759999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.63800000000000001},"max":{"1min":1.752,"5min":2.278,"15min":2.278},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.2090000000000001,"15min":1.206},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.5349999999999999,"5min":1.9399999999999999,"15min":2.2759999999999998},"last":1.4079999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.141,"15min":1.1719999999999999},"min":{"1min":0.79600000000000004,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":2.0910000000000002,"15min":2.7189999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.264,"15min":1.2589999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.639,"5min":2.375,"15min":2.375},"last":1.181}]},{"osd":7,"last update":"Sat Oct 7 23:29:28 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.3320000000000001,"15min":1.298},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.70199999999999996},"max":{"1min":1.6299999999999999,"5min":2.5030000000000001,"15min":2.524},"last":1.266},{"interface":"front","average":{"1min":1.155,"5min":1.2270000000000001,"15min":1.238},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.413,"5min":2.1030000000000002,"15min":2.3330000000000002},"last":1.1040000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.2130000000000001,"15min":2.5129999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.33900000000000002},"max":{"1min":1.093,"5min":1.1419999999999999,"15min":1.1890000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94199999999999995,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.452,"15min":1.6439999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.92600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.42299999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.5089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.0189999999999999,"15min":0.95699999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.42899999999999999},"max":{"1min":1.3440000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":1.798,"15min":1.798},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.046},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.478,"5min":1.603,"15min":1.603},"last":0.999}]},{"osd":5,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"max":{"1min":1.371,"5min":1.548,"15min":1.548},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.1719999999999999,"15min":1.1659999999999999},"min":{"1min":0.84299999999999997,"5min":0.745,"15min":0.53900000000000003},"max":{"1min":1.397,"5min":1.6859999999999999,"15min":1.8129999999999999},"last":0.876}]},{"osd":6,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.202},"min":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.57299999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.895},"last":1.071},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.22,"15min":1.2130000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.67200000000000004},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.802},"last":0.89900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.248,"15min":1.248},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.64500000000000002},"max":{"1min":1.3540000000000001,"5min":1.7809999999999999,"15min":1.8939999999999999},"last":1.022},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.27,"15min":1.27},"min":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.625},"max":{"1min":1.6930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:30:32.447 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-07T23:30:32.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 Processor -- start 2023-10-07T23:30:32.557 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 -- start start 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1354158be0 0x7f13540ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135406b3c0 con 0x7f135406b850 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135406a7b0 con 0x7f1354158fb0 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f13594b3700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135406aa90 con 0x7f1354158be0 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1354158be0 0x7f13540ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f1354158be0 0x7f13540ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40834/0 (socket says 172.21.15.57:40834) 2023-10-07T23:30:32.559 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 -- 172.21.15.57:0/826453326 learned_addr learned my addr 172.21.15.57:0/826453326 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 -- 172.21.15.57:0/826453326 >> v1:172.21.15.154:6789/0 conn(0x7f1354158fb0 legacy=0x7f13540cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 -- 172.21.15.57:0/826453326 >> v1:172.21.15.57:6789/0 conn(0x7f135406b850 legacy=0x7f13540ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1352ffd700 1 -- 172.21.15.57:0/826453326 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665850140 0 0) 0x7f135406b3c0 con 0x7f135406b850 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 -- 172.21.15.57:0/826453326 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135406ad70 con 0x7f1354158be0 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1353fff700 1 --2- 172.21.15.57:0/826453326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1354158be0 0x7f13540ce170 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f133c0099b0 tx=0x7f133c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ff25b5ba10904d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1352ffd700 1 -- 172.21.15.57:0/826453326 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00b860 con 0x7f1354158be0 2023-10-07T23:30:32.560 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1352ffd700 1 -- 172.21.15.57:0/826453326 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c00b9c0 con 0x7f1354158be0 2023-10-07T23:30:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.552+0000 7f1352ffd700 1 -- 172.21.15.57:0/826453326 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00bd60 con 0x7f1354158be0 2023-10-07T23:30:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- 172.21.15.57:0/826453326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1354158be0 msgr2=0x7f13540ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.561 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 --2- 172.21.15.57:0/826453326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1354158be0 0x7f13540ce170 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f133c0099b0 tx=0x7f133c009250 comp rx=0 tx=0).stop 2023-10-07T23:30:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- 172.21.15.57:0/826453326 shutdown_connections 2023-10-07T23:30:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 --2- 172.21.15.57:0/826453326 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1354158be0 0x7f13540ce170 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- 172.21.15.57:0/826453326 >> 172.21.15.57:0/826453326 conn(0x7f1354068520 msgr2=0x7f1354068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- 172.21.15.57:0/826453326 shutdown_connections 2023-10-07T23:30:32.562 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- 172.21.15.57:0/826453326 wait complete. 2023-10-07T23:30:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 Processor -- start 2023-10-07T23:30:32.563 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.556+0000 7f13594b3700 1 -- start start 2023-10-07T23:30:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.565 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1354158be0 0x7f135415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1354158fb0 0x7f135415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135415d130 con 0x7f135406b850 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135415cca0 con 0x7f1354158be0 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1354144d30 con 0x7f1354158fb0 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.566 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40846/0 (socket says 172.21.15.57:40846) 2023-10-07T23:30:32.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 -- 172.21.15.57:0/1899186366 learned_addr learned my addr 172.21.15.57:0/1899186366 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:32.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1358cb2700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1354158fb0 0x7f135415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.567 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13537fe700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1354158be0 0x7f135415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 -- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1354158fb0 msgr2=0x7f135415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1354158fb0 0x7f135415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 -- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1354158be0 msgr2=0x7f135415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1354158be0 0x7f135415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.568 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 -- 172.21.15.57:0/1899186366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0094f0 con 0x7f135406b850 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f1353fff700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f133c006f50 tx=0x7f133c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c01f070 con 0x7f135406b850 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c011bb0 con 0x7f135406b850 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135415c090 con 0x7f135406b850 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c025420 con 0x7f135406b850 2023-10-07T23:30:32.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.560+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135415c650 con 0x7f135406b850 2023-10-07T23:30:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13540711c0 con 0x7f135406b850 2023-10-07T23:30:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f133c023070 con 0x7f135406b850 2023-10-07T23:30:32.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f134bfff700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 0x7f1334062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f133c07d9b0 con 0x7f135406b850 2023-10-07T23:30:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f13537fe700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 0x7f1334062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:32.571 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.564+0000 7f13537fe700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 0x7f1334062a30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1354157320 tx=0x7f13440016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:32.578 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.572+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f133c04e270 con 0x7f135406b850 2023-10-07T23:30:32.741 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.736+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f135406d4c0 con 0x7f135406b850 2023-10-07T23:30:32.743 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.736+0000 7f134bfff700 1 -- 172.21.15.57:0/1899186366 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12324 (secure 0 0 0) 0x7f133c051890 con 0x7f135406b850 2023-10-07T23:30:32.743 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:32.744 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":53,"fsid":"aaf09e93-912b-470e-9df7-bb9a606f50e4","created":"2023-10-07T23:21:00.859142+0000","modified":"2023-10-07T23:30:28.596220+0000","last_up_change":"2023-10-07T23:21:11.807304+0000","last_in_change":"2023-10-07T23:21:02.935726+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-07T23:21:13.132066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"4dedf766-fa5e-4cc1-bef2-7238b3aa9a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":33391},{"type":"v1","addr":"172.21.15.57:6825","nonce":33391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":33391},{"type":"v1","addr":"172.21.15.57:6827","nonce":33391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":33391},{"type":"v1","addr":"172.21.15.57:6831","nonce":33391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":33391},{"type":"v1","addr":"172.21.15.57:6829","nonce":33391}]},"public_addr":"172.21.15.57:6825/33391","cluster_addr":"172.21.15.57:6827/33391","heartbeat_back_addr":"172.21.15.57:6831/33391","heartbeat_front_addr":"172.21.15.57:6829/33391","state":["exists","up"]},{"osd":1,"uuid":"ea9dba34-6392-410e-8ae1-e990532b8721","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":33405},{"type":"v1","addr":"172.21.15.57:6817","nonce":33405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":33405},{"type":"v1","addr":"172.21.15.57:6819","nonce":33405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":33405},{"type":"v1","addr":"172.21.15.57:6823","nonce":33405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":33405},{"type":"v1","addr":"172.21.15.57:6821","nonce":33405}]},"public_addr":"172.21.15.57:6817/33405","cluster_addr":"172.21.15.57:6819/33405","heartbeat_back_addr":"172.21.15.57:6823/33405","heartbeat_front_addr":"172.21.15.57:6821/33405","state":["exists","up"]},{"osd":2,"uuid":"b21b4335-8486-43d1-90ab-bc9f750452fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":33393},{"type":"v1","addr":"172.21.15.57:6801","nonce":33393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":33393},{"type":"v1","addr":"172.21.15.57:6803","nonce":33393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":33393},{"type":"v1","addr":"172.21.15.57:6807","nonce":33393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":33393},{"type":"v1","addr":"172.21.15.57:6805","nonce":33393}]},"public_addr":"172.21.15.57:6801/33393","cluster_addr":"172.21.15.57:6803/33393","heartbeat_back_addr":"172.21.15.57:6807/33393","heartbeat_front_addr":"172.21.15.57:6805/33393","state":["exists","up"]},{"osd":3,"uuid":"2ac0335f-8cc0-4c5c-a0dd-adb6700ab9f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":33394},{"type":"v1","addr":"172.21.15.57:6809","nonce":33394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":33394},{"type":"v1","addr":"172.21.15.57:6811","nonce":33394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":33394},{"type":"v1","addr":"172.21.15.57:6815","nonce":33394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":33394},{"type":"v1","addr":"172.21.15.57:6813","nonce":33394}]},"public_addr":"172.21.15.57:6809/33394","cluster_addr":"172.21.15.57:6811/33394","heartbeat_back_addr":"172.21.15.57:6815/33394","heartbeat_front_addr":"172.21.15.57:6813/33394","state":["exists","up"]},{"osd":4,"uuid":"d167c366-16e6-47d5-9be7-2700c4f4e71a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":33720},{"type":"v1","addr":"172.21.15.154:6809","nonce":33720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":33720},{"type":"v1","addr":"172.21.15.154:6811","nonce":33720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":33720},{"type":"v1","addr":"172.21.15.154:6817","nonce":33720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":33720},{"type":"v1","addr":"172.21.15.154:6814","nonce":33720}]},"public_addr":"172.21.15.154:6809/33720","cluster_addr":"172.21.15.154:6811/33720","heartbeat_back_addr":"172.21.15.154:6817/33720","heartbeat_front_addr":"172.21.15.154:6814/33720","state":["exists","up"]},{"osd":5,"uuid":"012ff59d-c3fd-48aa-9b9d-c4cab134c7aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":33759},{"type":"v1","addr":"172.21.15.154:6825","nonce":33759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":33759},{"type":"v1","addr":"172.21.15.154:6827","nonce":33759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":33759},{"type":"v1","addr":"172.21.15.154:6831","nonce":33759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":33759},{"type":"v1","addr":"172.21.15.154:6829","nonce":33759}]},"public_addr":"172.21.15.154:6825/33759","cluster_addr":"172.21.15.154:6827/33759","heartbeat_back_addr":"172.21.15.154:6831/33759","heartbeat_front_addr":"172.21.15.154:6829/33759","state":["exists","up"]},{"osd":6,"uuid":"18cd0831-a7c6-404a-af7b-ed249e03fcfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":33721},{"type":"v1","addr":"172.21.15.154:6801","nonce":33721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":33721},{"type":"v1","addr":"172.21.15.154:6803","nonce":33721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":33721},{"type":"v1","addr":"172.21.15.154:6807","nonce":33721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":33721},{"type":"v1","addr":"172.21.15.154:6805","nonce":33721}]},"public_addr":"172.21.15.154:6801/33721","cluster_addr":"172.21.15.154:6803/33721","heartbeat_back_addr":"172.21.15.154:6807/33721","heartbeat_front_addr":"172.21.15.154:6805/33721","state":["exists","up"]},{"osd":7,"uuid":"87479800-b391-4784-8fed-c0278af5f8dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":33722},{"type":"v1","addr":"172.21.15.154:6815","nonce":33722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":33722},{"type":"v1","addr":"172.21.15.154:6819","nonce":33722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":33722},{"type":"v1","addr":"172.21.15.154:6823","nonce":33722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":33722},{"type":"v1","addr":"172.21.15.154:6821","nonce":33722}]},"public_addr":"172.21.15.154:6815/33722","cluster_addr":"172.21.15.154:6819/33722","heartbeat_back_addr":"172.21.15.154:6823/33722","heartbeat_front_addr":"172.21.15.154:6821/33722","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.449328+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.048706+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.212420+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.272779+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.165371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.453244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.424584+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-07T23:21:08.430630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/320776638":"2023-10-08T23:30:25.586325+0000","172.21.15.154:6834/320776638":"2023-10-08T23:30:25.586325+0000","192.168.0.1:0/1259509611":"2023-10-08T00:30:19.028952+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-07T23:30:32.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 msgr2=0x7f1334062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 0x7f1334062a30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1354157320 tx=0x7f13440016f0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.745 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 msgr2=0x7f13541447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f133c006f50 tx=0x7f133c017000 comp rx=0 tx=0).stop 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 shutdown_connections 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1354158fb0 0x7f135415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1334060580 0x7f1334062a30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1354158be0 0x7f135415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 --2- 172.21.15.57:0/1899186366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f135406b850 0x7f13541447f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:32.746 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 >> 172.21.15.57:0/1899186366 conn(0x7f1354068520 msgr2=0x7f1354157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 shutdown_connections 2023-10-07T23:30:32.747 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:32.740+0000 7f13594b3700 1 -- 172.21.15.57:0/1899186366 wait complete. 2023-10-07T23:30:33.759 INFO:tasks.ceph:Scrubbing osd.0 2023-10-07T23:30:33.759 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:33.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.875+0000 7f7fb50c0700 1 Processor -- start 2023-10-07T23:30:33.882 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.875+0000 7f7fb50c0700 1 -- start start 2023-10-07T23:30:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 0x7f7fb00c3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00ca440 con 0x7f7fb00cbe70 2023-10-07T23:30:33.885 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00c38d0 con 0x7f7fb0158f10 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb50c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00c9fb0 con 0x7f7fb0158b40 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb48bf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb48bf700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40852/0 (socket says 172.21.15.57:40852) 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:41246/0 (socket says 172.21.15.57:41246) 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 -- 172.21.15.57:0/1251849579 learned_addr learned my addr 172.21.15.57:0/1251849579 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:33.886 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faf7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 0x7f7fb00c3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 -- 172.21.15.57:0/1251849579 >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 msgr2=0x7f7fb00c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 0x7f7fb00c3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.888 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 -- 172.21.15.57:0/1251849579 >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 msgr2=0x7f7fb00cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 -- 172.21.15.57:0/1251849579 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb00c93e0 con 0x7f7fb0158b40 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faf7fe700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 0x7f7fb00c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7fb48bf700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faffff700 1 --2- 172.21.15.57:0/1251849579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fb006ac90 tx=0x7f7fa0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4663d0a36cd6ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faeffd700 1 -- 172.21.15.57:0/1251849579 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000b640 con 0x7f7fb0158b40 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faeffd700 1 -- 172.21.15.57:0/1251849579 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000b7a0 con 0x7f7fb0158b40 2023-10-07T23:30:33.889 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.879+0000 7f7faeffd700 1 -- 172.21.15.57:0/1251849579 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000bb60 con 0x7f7fb0158b40 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- 172.21.15.57:0/1251849579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 msgr2=0x7f7fb00c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/1251849579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fb006ac90 tx=0x7f7fa0009250 comp rx=0 tx=0).stop 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- 172.21.15.57:0/1251849579 shutdown_connections 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.57:3300/0 conn(0x7f7fb00cbe70 0x7f7fb00cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/1251849579 >> v2:172.21.15.154:3300/0 conn(0x7f7fb0158f10 0x7f7fb00c3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/1251849579 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00c2e50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.890 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- 172.21.15.57:0/1251849579 >> 172.21.15.57:0/1251849579 conn(0x7f7fb0095d30 msgr2=0x7f7fb006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- 172.21.15.57:0/1251849579 shutdown_connections 2023-10-07T23:30:33.891 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- 172.21.15.57:0/1251849579 wait complete. 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 Processor -- start 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- start start 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 0x7f7fb00af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb0158f10 0x7f7fb00b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.893 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0153500 con 0x7f7fb00cbe70 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0169ef0 con 0x7f7fb0158f10 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb50c0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00b1ac0 con 0x7f7fb0158b40 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7faffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 0x7f7fb00af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7faffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 0x7f7fb00af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40860/0 (socket says 172.21.15.57:40860) 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7faffff700 1 -- 172.21.15.57:0/3703134507 learned_addr learned my addr 172.21.15.57:0/3703134507 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7faf7fe700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.883+0000 7f7fb48bf700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb0158f10 0x7f7fb00b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.894 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 -- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 msgr2=0x7f7fb00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7fb0158b40 0x7f7fb00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 -- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb0158f10 msgr2=0x7f7fb00b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fb0158f10 0x7f7fb00b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009a20 con 0x7f7fb00cbe70 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7faffff700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 0x7f7fb00af860 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0006fd0 tx=0x7f7fa0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0000d10 con 0x7f7fb00cbe70 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000bb20 con 0x7f7fb00cbe70 2023-10-07T23:30:33.895 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4009420 con 0x7f7fb00cbe70 2023-10-07T23:30:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0011630 con 0x7f7fb00cbe70 2023-10-07T23:30:33.896 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa00094f0 con 0x7f7fb00cbe70 2023-10-07T23:30:33.897 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fb00630c0 con 0x7f7fb00cbe70 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f7fa00119a0 con 0x7f7fb00cbe70 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f9c060550 0x7f7f9c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.887+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f7fa007de70 con 0x7f7fb00cbe70 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7faf7fe700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f9c060550 0x7f7f9c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7f97fff700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7f9c066280 0x7f7f9c068690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f9c068d40 con 0x7f7f9c066280 2023-10-07T23:30:33.898 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fa007e240 con 0x7f7fb00cbe70 2023-10-07T23:30:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7fb48bf700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7f9c066280 0x7f7f9c068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7fb48bf700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7f9c066280 0x7f7f9c068690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7faf7fe700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f9c060550 0x7f7f9c062a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ae830 tx=0x7f7fa40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:33.899 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.891+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== osd.0 v2:172.21.15.57:6824/33391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7f9c068d40 con 0x7f7f9c066280 2023-10-07T23:30:33.911 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.903+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 --> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7fb00711c0 con 0x7f7f9c066280 2023-10-07T23:30:33.912 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7f97fff700 1 -- 172.21.15.57:0/3703134507 <== osd.0 v2:172.21.15.57:6824/33391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7fb00711c0 con 0x7f7f9c066280 2023-10-07T23:30:33.912 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:33.912 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:33.912 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:33.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7f9c066280 msgr2=0x7f7f9c068690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:6824/33391,v1:172.21.15.57:6825/33391] conn(0x7f7f9c066280 0x7f7f9c068690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.913 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f9c060550 msgr2=0x7f7f9c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7f9c060550 0x7f7f9c062a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ae830 tx=0x7f7fa40091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 msgr2=0x7f7fb00af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 --2- 172.21.15.57:0/3703134507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7fb00cbe70 0x7f7fb00af860 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0006fd0 tx=0x7f7fa0017000 comp rx=0 tx=0).stop 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb48bf700 1 -- 172.21.15.57:0/3703134507 reap_dead start 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 shutdown_connections 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 >> 172.21.15.57:0/3703134507 conn(0x7f7fb0095d30 msgr2=0x7f7fb01558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:33.914 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 shutdown_connections 2023-10-07T23:30:33.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:33.907+0000 7f7fb50c0700 1 -- 172.21.15.57:0/3703134507 wait complete. 2023-10-07T23:30:33.924 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-07T23:30:34.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 Processor -- start 2023-10-07T23:30:34.029 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 -- start start 2023-10-07T23:30:34.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa39c158d70 0x7fa39c159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 0x7fa39c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c06b1a0 con 0x7fa39c06cd40 2023-10-07T23:30:34.031 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c0c7620 con 0x7fa39c158d70 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a2a8b700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c06d1c0 con 0x7fa39c06b630 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa39c158d70 0x7fa39c159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa39c158d70 0x7fa39c159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40870/0 (socket says 172.21.15.57:40870) 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 -- 172.21.15.57:0/2537992643 learned_addr learned my addr 172.21.15.57:0/2537992643 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa39bfff700 1 --2- 172.21.15.57:0/2537992643 >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 0x7fa39c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 -- 172.21.15.57:0/2537992643 >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 msgr2=0x7fa39c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:34.032 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 --2- 172.21.15.57:0/2537992643 >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 0x7fa39c06c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 -- 172.21.15.57:0/2537992643 >> v1:172.21.15.154:6789/0 conn(0x7fa39c06cd40 legacy=0x7fa39c0c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 -- 172.21.15.57:0/2537992643 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c06ba80 con 0x7fa39c158d70 2023-10-07T23:30:34.033 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa39bfff700 1 --2- 172.21.15.57:0/2537992643 >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 0x7fa39c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:34.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa3a0827700 1 --2- 172.21.15.57:0/2537992643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c158d70 0x7fa39c159140 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06a4d0 tx=0x7fa38c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f71f7750894217 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.023+0000 7fa39b7fe700 1 -- 172.21.15.57:0/2537992643 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00b820 con 0x7fa39c158d70 2023-10-07T23:30:34.034 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa39b7fe700 1 -- 172.21.15.57:0/2537992643 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c00b980 con 0x7fa39c158d70 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa39b7fe700 1 -- 172.21.15.57:0/2537992643 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c00bd20 con 0x7fa39c158d70 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2537992643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c158d70 msgr2=0x7fa39c159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2537992643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c158d70 0x7fa39c159140 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06a4d0 tx=0x7fa38c009250 comp rx=0 tx=0).stop 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2537992643 shutdown_connections 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2537992643 >> v2:172.21.15.154:3300/0 conn(0x7fa39c06b630 0x7fa39c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2537992643 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c158d70 0x7fa39c159140 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.035 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2537992643 >> 172.21.15.57:0/2537992643 conn(0x7fa39c062ba0 msgr2=0x7fa39c062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2537992643 shutdown_connections 2023-10-07T23:30:34.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2537992643 wait complete. 2023-10-07T23:30:34.036 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 Processor -- start 2023-10-07T23:30:34.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.027+0000 7fa3a2a8b700 1 -- start start 2023-10-07T23:30:34.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa39c06b630 0x7fa39c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.038 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 0x7fa39c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c157350 con 0x7fa39c06cd40 2023-10-07T23:30:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1577e0 con 0x7fa39c1511a0 2023-10-07T23:30:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c151640 con 0x7fa39c06b630 2023-10-07T23:30:34.039 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40872/0 (socket says 172.21.15.57:40872) 2023-10-07T23:30:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa39bfff700 1 -- 172.21.15.57:0/2548173431 learned_addr learned my addr 172.21.15.57:0/2548173431 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a0827700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa39c06b630 0x7fa39c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.040 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 0x7fa39c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 -- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa39c06b630 msgr2=0x7fa39c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa39c06b630 0x7fa39c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 -- 172.21.15.57:0/2548173431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 msgr2=0x7fa39c0c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.041 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 -- 172.21.15.57:0/2548173431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c0094f0 con 0x7fa39c1511a0 2023-10-07T23:30:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa39bfff700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:34.042 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a1028700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 0x7fa39c0ca7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c830 tx=0x7fa394001690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.043 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394012630 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa394012790 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c156740 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394012b50 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.031+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c156d00 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c0647b0 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa394011380 con 0x7fa39c1511a0 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa398ff9700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 0x7fa388062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.044 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fa39407c650 con 0x7fa39c1511a0 2023-10-07T23:30:34.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa3a0827700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 0x7fa388062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.045 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.035+0000 7fa3a0827700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 0x7fa388062a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06a4d0 tx=0x7fa38c017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.049 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.043+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa39404cf10 con 0x7fa39c1511a0 2023-10-07T23:30:34.214 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.207+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39c0b1650 con 0x7fa388060550 2023-10-07T23:30:34.215 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.207+0000 7fa398ff9700 1 -- 172.21.15.57:0/2548173431 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa39c0b1650 con 0x7fa388060550 2023-10-07T23:30:34.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 msgr2=0x7fa388062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 0x7fa388062a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06a4d0 tx=0x7fa38c017000 comp rx=0 tx=0).stop 2023-10-07T23:30:34.219 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 msgr2=0x7fa39c0ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 0x7fa39c0ca7d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa39400c830 tx=0x7fa394001690 comp rx=0 tx=0).stop 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 shutdown_connections 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa388060550 0x7fa388062a00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa39c1511a0 0x7fa39c0ca7d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa39c06cd40 0x7fa39c0c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 --2- 172.21.15.57:0/2548173431 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa39c06b630 0x7fa39c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 >> 172.21.15.57:0/2548173431 conn(0x7fa39c062ba0 msgr2=0x7fa39c15c150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.220 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 shutdown_connections 2023-10-07T23:30:34.221 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.211+0000 7fa3a2a8b700 1 -- 172.21.15.57:0/2548173431 wait complete. 2023-10-07T23:30:34.221 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 0 to deep-scrub 2023-10-07T23:30:34.235 INFO:tasks.ceph:Scrubbing osd.1 2023-10-07T23:30:34.235 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:34.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.343+0000 7f84418a9700 1 Processor -- start 2023-10-07T23:30:34.351 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.343+0000 7f84418a9700 1 -- start start 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c156f30 0x7f843c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f843c157300 0x7f843c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c06d470 con 0x7f843c0c9670 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c06c8a0 con 0x7f843c157300 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c06cb40 con 0x7f843c156f30 2023-10-07T23:30:34.354 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f843c157300 0x7f843c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f843c157300 0x7f843c0bfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40888/0 (socket says 172.21.15.57:40888) 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 -- 172.21.15.57:0/3491498381 learned_addr learned my addr 172.21.15.57:0/3491498381 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f843affd700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c156f30 0x7f843c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 -- 172.21.15.57:0/3491498381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c156f30 msgr2=0x7f843c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c156f30 0x7f843c06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 -- 172.21.15.57:0/3491498381 >> v1:172.21.15.154:6789/0 conn(0x7f843c0c9670 legacy=0x7f843c0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 -- 172.21.15.57:0/3491498381 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c06ce20 con 0x7f843c157300 2023-10-07T23:30:34.355 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f8433fff700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0bfec0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84240099b0 tx=0x7f8424009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cda11e30e3354d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f843a7fc700 1 -- 172.21.15.57:0/3491498381 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400b860 con 0x7f843c157300 2023-10-07T23:30:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f843a7fc700 1 -- 172.21.15.57:0/3491498381 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842400b9c0 con 0x7f843c157300 2023-10-07T23:30:34.356 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f843a7fc700 1 -- 172.21.15.57:0/3491498381 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400bd60 con 0x7f843c157300 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- 172.21.15.57:0/3491498381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 msgr2=0x7f843c0bfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0bfec0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84240099b0 tx=0x7f8424009250 comp rx=0 tx=0).stop 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- 172.21.15.57:0/3491498381 shutdown_connections 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0bfec0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 --2- 172.21.15.57:0/3491498381 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c156f30 0x7f843c06d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.357 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- 172.21.15.57:0/3491498381 >> 172.21.15.57:0/3491498381 conn(0x7f843c0ccf10 msgr2=0x7f843c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- 172.21.15.57:0/3491498381 shutdown_connections 2023-10-07T23:30:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.347+0000 7f84418a9700 1 -- 172.21.15.57:0/3491498381 wait complete. 2023-10-07T23:30:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 Processor -- start 2023-10-07T23:30:34.358 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- start start 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c0c9670 0x7f843c0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f843c156f30 0x7f843c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c156520 con 0x7f843c157300 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c156090 con 0x7f843c156f30 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1551e0 con 0x7f843c0c9670 2023-10-07T23:30:34.360 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40900/0 (socket says 172.21.15.57:40900) 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 -- 172.21.15.57:0/4043546149 learned_addr learned my addr 172.21.15.57:0/4043546149 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c0c9670 0x7f843c0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f8433fff700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f843c156f30 0x7f843c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 -- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c0c9670 msgr2=0x7f843c0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f843c0c9670 0x7f843c0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.361 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 -- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f843c156f30 msgr2=0x7f843c154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f843c156f30 0x7f843c154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84240094f0 con 0x7f843c157300 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f843b7fe700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0ca670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f843400c910 tx=0x7f84340016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434012630 con 0x7f843c157300 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434012790 con 0x7f843c157300 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c155480 con 0x7f843c157300 2023-10-07T23:30:34.362 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434012b30 con 0x7f843c157300 2023-10-07T23:30:34.363 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.351+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c155a40 con 0x7f843c157300 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.355+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f843c0647b0 con 0x7f843c157300 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.355+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f843401a3a0 con 0x7f843c157300 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.355+0000 7f8432ffd700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f841c060550 0x7f841c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.355+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f843407c7b0 con 0x7f843c157300 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.355+0000 7f8432ffd700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7f841c0663f0 0x7f841c068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.364 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f841c068eb0 con 0x7f841c0663f0 2023-10-07T23:30:34.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f843affd700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f841c060550 0x7f841c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f843407ed10 con 0x7f843c157300 2023-10-07T23:30:34.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f8433fff700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7f841c0663f0 0x7f841c068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.365 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f8433fff700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7f841c0663f0 0x7f841c068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f843affd700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f841c060550 0x7f841c062a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f842c006fd0 tx=0x7f842c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.366 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.359+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== osd.1 v2:172.21.15.57:6816/33405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f841c068eb0 con 0x7f841c0663f0 2023-10-07T23:30:34.379 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.371+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 --> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f843c06d100 con 0x7f841c0663f0 2023-10-07T23:30:34.380 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f8432ffd700 1 -- 172.21.15.57:0/4043546149 <== osd.1 v2:172.21.15.57:6816/33405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f843c06d100 con 0x7f841c0663f0 2023-10-07T23:30:34.380 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:34.381 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:34.381 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:34.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7f841c0663f0 msgr2=0x7f841c068800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:6816/33405,v1:172.21.15.57:6817/33405] conn(0x7f841c0663f0 0x7f841c068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f841c060550 msgr2=0x7f841c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f841c060550 0x7f841c062a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f842c006fd0 tx=0x7f842c008040 comp rx=0 tx=0).stop 2023-10-07T23:30:34.382 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 msgr2=0x7f843c0ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 --2- 172.21.15.57:0/4043546149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f843c157300 0x7f843c0ca670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f843400c910 tx=0x7f84340016b0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f843b7fe700 1 -- 172.21.15.57:0/4043546149 reap_dead start 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 shutdown_connections 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 >> 172.21.15.57:0/4043546149 conn(0x7f843c0ccf10 msgr2=0x7f843c1525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 shutdown_connections 2023-10-07T23:30:34.383 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.375+0000 7f84418a9700 1 -- 172.21.15.57:0/4043546149 wait complete. 2023-10-07T23:30:34.397 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-07T23:30:34.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 Processor -- start 2023-10-07T23:30:34.509 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- start start 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f19b4161a70 0x7f19b406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b406bbf0 con 0x7f19b415d2a0 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b406b020 con 0x7f19b415ced0 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b40c9010 con 0x7f19b4161a70 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b2ffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f19b4161a70 0x7f19b406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.510 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b3fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f19b415d2a0 0x7f19b40c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38892/0 (socket says 172.21.15.57:38892) 2023-10-07T23:30:34.511 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b3fff700 1 -- 172.21.15.57:0/880378486 learned_addr learned my addr 172.21.15.57:0/880378486 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b2ffd700 1 -- 172.21.15.57:0/880378486 >> v1:172.21.15.154:6789/0 conn(0x7f19b415ced0 legacy=0x7f19b40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b2ffd700 1 -- 172.21.15.57:0/880378486 >> v1:172.21.15.57:6789/0 conn(0x7f19b415d2a0 legacy=0x7f19b40c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.512 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b2ffd700 1 -- 172.21.15.57:0/880378486 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b406b2c0 con 0x7f19b4161a70 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b2ffd700 1 --2- 172.21.15.57:0/880378486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b4161a70 0x7f19b406c080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009de0 tx=0x7f19a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e3b01779235e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b27fc700 1 -- 172.21.15.57:0/880378486 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400e070 con 0x7f19b4161a70 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b27fc700 1 -- 172.21.15.57:0/880378486 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a400bbb0 con 0x7f19b4161a70 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b27fc700 1 -- 172.21.15.57:0/880378486 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4012430 con 0x7f19b4161a70 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- 172.21.15.57:0/880378486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b4161a70 msgr2=0x7f19b406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 --2- 172.21.15.57:0/880378486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b4161a70 0x7f19b406c080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009de0 tx=0x7f19a4009780 comp rx=0 tx=0).stop 2023-10-07T23:30:34.513 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 -- 172.21.15.57:0/880378486 shutdown_connections 2023-10-07T23:30:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.503+0000 7f19b8a24700 1 --2- 172.21.15.57:0/880378486 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b4161a70 0x7f19b406c080 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- 172.21.15.57:0/880378486 >> 172.21.15.57:0/880378486 conn(0x7f19b40670e0 msgr2=0x7f19b40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- 172.21.15.57:0/880378486 shutdown_connections 2023-10-07T23:30:34.514 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- 172.21.15.57:0/880378486 wait complete. 2023-10-07T23:30:34.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 Processor -- start 2023-10-07T23:30:34.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- start start 2023-10-07T23:30:34.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 0x7f19b4157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19b415d2a0 0x7f19b40ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b415e730 con 0x7f19b415ced0 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b415ea10 con 0x7f19b4161a70 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b8a24700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b415ecf0 con 0x7f19b415d2a0 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 0x7f19b4157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.517 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.57:60044/0 (socket says 172.21.15.57:60044) 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b37fe700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 0x7f19b4157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40920/0 (socket says 172.21.15.57:40920) 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 -- 172.21.15.57:0/549642010 learned_addr learned my addr 172.21.15.57:0/549642010 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b2ffd700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19b415d2a0 0x7f19b40ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 -- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19b415d2a0 msgr2=0x7f19b40ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19b415d2a0 0x7f19b40ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 -- 172.21.15.57:0/549642010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 msgr2=0x7f19b4157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 0x7f19b4157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.507+0000 7f19b3fff700 1 -- 172.21.15.57:0/549642010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4009a20 con 0x7f19b4161a70 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19b3fff700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c560 tx=0x7f19ac007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac005b30 con 0x7f19b4161a70 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac005c90 con 0x7f19b4161a70 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b415efd0 con 0x7f19b4161a70 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0072e0 con 0x7f19b4161a70 2023-10-07T23:30:34.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b40af4f0 con 0x7f19b4161a70 2023-10-07T23:30:34.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b40711c0 con 0x7f19b4161a70 2023-10-07T23:30:34.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f19ac007690 con 0x7f19b4161a70 2023-10-07T23:30:34.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.511+0000 7f19ab7fe700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 0x7f1994062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.515+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f19ac07b660 con 0x7f19b4161a70 2023-10-07T23:30:34.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.515+0000 7f19b37fe700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 0x7f1994062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.521 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.515+0000 7f19b37fe700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 0x7f1994062a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f199c000f60 tx=0x7f199c007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.529 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.523+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mon.1 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19ac049be0 con 0x7f19b4161a70 2023-10-07T23:30:34.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f19b415eb50 con 0x7f1994060550 2023-10-07T23:30:34.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19ab7fe700 1 -- 172.21.15.57:0/549642010 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f19b415eb50 con 0x7f1994060550 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 msgr2=0x7f1994062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 0x7f1994062a00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f199c000f60 tx=0x7f199c007ea0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 msgr2=0x7f19b40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00c560 tx=0x7f19ac007f60 comp rx=0 tx=0).stop 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 shutdown_connections 2023-10-07T23:30:34.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f1994060550 0x7f1994062a00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19b4161a70 0x7f19b40aeda0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f19b415d2a0 0x7f19b40ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 --2- 172.21.15.57:0/549642010 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19b415ced0 0x7f19b4157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 >> 172.21.15.57:0/549642010 conn(0x7f19b40670e0 msgr2=0x7f19b40c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.687+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 shutdown_connections 2023-10-07T23:30:34.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.691+0000 7f19b8a24700 1 -- 172.21.15.57:0/549642010 wait complete. 2023-10-07T23:30:34.698 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 1 to deep-scrub 2023-10-07T23:30:34.709 INFO:tasks.ceph:Scrubbing osd.2 2023-10-07T23:30:34.710 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:34.824 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 Processor -- start 2023-10-07T23:30:34.825 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 -- start start 2023-10-07T23:30:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb3f006dad0 0x7fb3f0157c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3f006dea0 0x7fb3f015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f015b800 con 0x7fb3f00c93c0 2023-10-07T23:30:34.829 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f015b240 con 0x7fb3f006dad0 2023-10-07T23:30:34.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ef59e700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f015b520 con 0x7fb3f006dea0 2023-10-07T23:30:34.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb3f006dad0 0x7fb3f0157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb3f006dad0 0x7fb3f0157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40928/0 (socket says 172.21.15.57:40928) 2023-10-07T23:30:34.830 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3eed9d700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fb3f00c93c0 0x7fb3f00c9800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38898/0 (socket says 172.21.15.57:38898) 2023-10-07T23:30:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 -- 172.21.15.57:0/870864760 learned_addr learned my addr 172.21.15.57:0/870864760 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3edd9b700 1 --2- 172.21.15.57:0/870864760 >> v2:172.21.15.154:3300/0 conn(0x7fb3f006dea0 0x7fb3f015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ed59a700 1 -- 172.21.15.57:0/870864760 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252566328 0 0) 0x7fb3f015b800 con 0x7fb3f00c93c0 2023-10-07T23:30:34.831 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ed59a700 1 -- 172.21.15.57:0/870864760 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003570 con 0x7fb3f00c93c0 2023-10-07T23:30:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 -- 172.21.15.57:0/870864760 >> v2:172.21.15.154:3300/0 conn(0x7fb3f006dea0 msgr2=0x7fb3f015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 --2- 172.21.15.57:0/870864760 >> v2:172.21.15.154:3300/0 conn(0x7fb3f006dea0 0x7fb3f015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 -- 172.21.15.57:0/870864760 >> v1:172.21.15.57:6789/0 conn(0x7fb3f00c93c0 legacy=0x7fb3f00c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.832 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 -- 172.21.15.57:0/870864760 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0157470 con 0x7fb3f006dad0 2023-10-07T23:30:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.819+0000 7fb3ee59c700 1 --2- 172.21.15.57:0/870864760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f006dad0 0x7fb3f0157c70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cc1f0 tx=0x7fb3e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ee7d7db7292e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ed59a700 1 -- 172.21.15.57:0/870864760 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000b640 con 0x7fb3f006dad0 2023-10-07T23:30:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ed59a700 1 -- 172.21.15.57:0/870864760 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000b7a0 con 0x7fb3f006dad0 2023-10-07T23:30:34.833 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ed59a700 1 -- 172.21.15.57:0/870864760 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000bb60 con 0x7fb3f006dad0 2023-10-07T23:30:34.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- 172.21.15.57:0/870864760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f006dad0 msgr2=0x7fb3f0157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/870864760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f006dad0 0x7fb3f0157c70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cc1f0 tx=0x7fb3e0009250 comp rx=0 tx=0).stop 2023-10-07T23:30:34.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- 172.21.15.57:0/870864760 shutdown_connections 2023-10-07T23:30:34.836 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/870864760 >> v2:172.21.15.154:3300/0 conn(0x7fb3f006dea0 0x7fb3f015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/870864760 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f006dad0 0x7fb3f0157c70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- 172.21.15.57:0/870864760 >> 172.21.15.57:0/870864760 conn(0x7fb3f0067b40 msgr2=0x7fb3f00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- 172.21.15.57:0/870864760 shutdown_connections 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- 172.21.15.57:0/870864760 wait complete. 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 Processor -- start 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.823+0000 7fb3ef59e700 1 -- start start 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3f006dad0 0x7fb3f016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.837 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3f006dea0 0x7fb3f0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 0x7fb3f015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f014aba0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f016a5e0 con 0x7fb3f006dad0 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f016a8c0 con 0x7fb3f006dea0 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 0x7fb3f015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.838 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 0x7fb3f015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40942/0 (socket says 172.21.15.57:40942) 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 -- 172.21.15.57:0/36806576 learned_addr learned my addr 172.21.15.57:0/36806576 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3edd9b700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3f006dea0 0x7fb3f0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ee59c700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3f006dad0 0x7fb3f016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 -- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3f006dea0 msgr2=0x7fb3f0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb3f006dea0 0x7fb3f0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 -- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3f006dad0 msgr2=0x7fb3f016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.839 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3f006dad0 0x7fb3f016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00094f0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3eed9d700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 0x7fb3f015cb30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800cd30 tx=0x7fb3e8001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800ce90 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e8005df0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f016aba0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e800aac0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.827+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0159bd0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.840 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3f00711c0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb3e80070f0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb3d0060550 0x7fb3d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fb3e807c590 con 0x7fb3f00c93c0 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3ee59c700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb3d0060550 0x7fb3d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7fb3d0066280 0x7fb3d0068690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.842 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3d0068d40 con 0x7fb3d0066280 2023-10-07T23:30:34.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3e807ebd0 con 0x7fb3f00c93c0 2023-10-07T23:30:34.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3edd9b700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7fb3d0066280 0x7fb3d0068690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3ee59c700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb3d0060550 0x7fb3d0062a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cc1f0 tx=0x7fb3e0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.831+0000 7fb3edd9b700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7fb3d0066280 0x7fb3d0068690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.843 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.835+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== osd.2 v2:172.21.15.57:6800/33393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb3d0068d40 con 0x7fb3d0066280 2023-10-07T23:30:34.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 --> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb3f014a830 con 0x7fb3d0066280 2023-10-07T23:30:34.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3de7fc700 1 -- 172.21.15.57:0/36806576 <== osd.2 v2:172.21.15.57:6800/33393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb3f014a830 con 0x7fb3d0066280 2023-10-07T23:30:34.861 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:34.861 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:34.861 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:34.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7fb3d0066280 msgr2=0x7fb3d0068690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:6800/33393,v1:172.21.15.57:6801/33393] conn(0x7fb3d0066280 0x7fb3d0068690 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.862 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb3d0060550 msgr2=0x7fb3d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb3d0060550 0x7fb3d0062a00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00cc1f0 tx=0x7fb3e0017000 comp rx=0 tx=0).stop 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 msgr2=0x7fb3f015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 --2- 172.21.15.57:0/36806576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb3f00c93c0 0x7fb3f015cb30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800cd30 tx=0x7fb3e8001b60 comp rx=0 tx=0).stop 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3eed9d700 1 -- 172.21.15.57:0/36806576 reap_dead start 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 shutdown_connections 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 >> 172.21.15.57:0/36806576 conn(0x7fb3f0067b40 msgr2=0x7fb3f00ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 shutdown_connections 2023-10-07T23:30:34.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.855+0000 7fb3ef59e700 1 -- 172.21.15.57:0/36806576 wait complete. 2023-10-07T23:30:34.879 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-07T23:30:34.990 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.983+0000 7f89e53da700 1 Processor -- start 2023-10-07T23:30:34.991 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.983+0000 7f89e53da700 1 -- start start 2023-10-07T23:30:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f89e0158e60 0x7f89e006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e00caba0 con 0x7f89e006c800 2023-10-07T23:30:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e006bb20 con 0x7f89e0159230 2023-10-07T23:30:34.992 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e006b560 con 0x7f89e0158e60 2023-10-07T23:30:34.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89df7fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f89e006c800 0x7f89e00cb030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38900/0 (socket says 172.21.15.57:38900) 2023-10-07T23:30:34.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89df7fe700 1 -- 172.21.15.57:0/3244222651 learned_addr learned my addr 172.21.15.57:0/3244222651 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:34.993 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89deffd700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f89e0158e60 0x7f89e006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574599115 0 0) 0x7f89e00caba0 con 0x7f89e006c800 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c8003570 con 0x7f89e006c800 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707674643 0 0) 0x7f89e006bb20 con 0x7f89e0159230 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e00caba0 con 0x7f89e0159230 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89deffd700 1 -- 172.21.15.57:0/3244222651 >> v1:172.21.15.154:6789/0 conn(0x7f89e0159230 legacy=0x7f89e00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.994 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89deffd700 1 -- 172.21.15.57:0/3244222651 >> v1:172.21.15.57:6789/0 conn(0x7f89e006c800 legacy=0x7f89e00cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89deffd700 1 -- 172.21.15.57:0/3244222651 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e006b840 con 0x7f89e0158e60 2023-10-07T23:30:34.995 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89deffd700 1 --2- 172.21.15.57:0/3244222651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e006abc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89d40098e0 tx=0x7f89d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138b3501625d6ca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4011630 con 0x7f89e0158e60 2023-10-07T23:30:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4011790 con 0x7f89e0158e60 2023-10-07T23:30:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89ddffb700 1 -- 172.21.15.57:0/3244222651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4011b30 con 0x7f89e0158e60 2023-10-07T23:30:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- 172.21.15.57:0/3244222651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 msgr2=0x7f89e006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:34.996 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 --2- 172.21.15.57:0/3244222651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e006abc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89d40098e0 tx=0x7f89d40091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- 172.21.15.57:0/3244222651 shutdown_connections 2023-10-07T23:30:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 --2- 172.21.15.57:0/3244222651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e006abc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- 172.21.15.57:0/3244222651 >> 172.21.15.57:0/3244222651 conn(0x7f89e0068740 msgr2=0x7f89e0068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- 172.21.15.57:0/3244222651 shutdown_connections 2023-10-07T23:30:34.997 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.987+0000 7f89e53da700 1 -- 172.21.15.57:0/3244222651 wait complete. 2023-10-07T23:30:34.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 Processor -- start 2023-10-07T23:30:34.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 -- start start 2023-10-07T23:30:34.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:34.999 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89e0159230 0x7f89e008a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0089900 con 0x7f89e0158e60 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0089b80 con 0x7f89e0159230 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89e53da700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e0089e60 con 0x7f89e006c800 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89de7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:41314/0 (socket says 172.21.15.57:41314) 2023-10-07T23:30:35.000 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89df7fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89e0159230 0x7f89e008a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 -- 172.21.15.57:0/412408301 learned_addr learned my addr 172.21.15.57:0/412408301 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 -- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89e0159230 msgr2=0x7f89e008a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89e0159230 0x7f89e008a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 -- 172.21.15.57:0/412408301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 msgr2=0x7f89e00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e00c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 -- 172.21.15.57:0/412408301 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009420 con 0x7f89e006c800 2023-10-07T23:30:35.001 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89de7fc700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:35.002 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89deffd700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e00644a0 tx=0x7f89d4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.991+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d401f070 con 0x7f89e006c800 2023-10-07T23:30:35.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4000bc0 con 0x7f89e006c800 2023-10-07T23:30:35.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e008a140 con 0x7f89e006c800 2023-10-07T23:30:35.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4025420 con 0x7f89e006c800 2023-10-07T23:30:35.003 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e00c87c0 con 0x7f89e006c800 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e008a640 con 0x7f89e006c800 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f89d4023070 con 0x7f89e006c800 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89c6ffd700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 0x7f89cc062ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f89d407d7e0 con 0x7f89e006c800 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.995+0000 7f89de7fc700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 0x7f89cc062ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.004 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:34.999+0000 7f89de7fc700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 0x7f89cc062ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cedd0 tx=0x7f89d0007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.007+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f89d404e000 con 0x7f89e006c800 2023-10-07T23:30:35.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.171+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f89e0089cf0 con 0x7f89cc060600 2023-10-07T23:30:35.179 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.171+0000 7f89c6ffd700 1 -- 172.21.15.57:0/412408301 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f89e0089cf0 con 0x7f89cc060600 2023-10-07T23:30:35.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 msgr2=0x7f89cc062ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 0x7f89cc062ab0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cedd0 tx=0x7f89d0007f90 comp rx=0 tx=0).stop 2023-10-07T23:30:35.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 msgr2=0x7f89e0155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89e00644a0 tx=0x7f89d4017000 comp rx=0 tx=0).stop 2023-10-07T23:30:35.182 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 shutdown_connections 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f89cc060600 0x7f89cc062ab0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89e0159230 0x7f89e008a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e0158e60 0x7f89e00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 --2- 172.21.15.57:0/412408301 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f89e006c800 0x7f89e0155850 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 >> 172.21.15.57:0/412408301 conn(0x7f89e0068740 msgr2=0x7f89e00cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 shutdown_connections 2023-10-07T23:30:35.183 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.175+0000 7f89e53da700 1 -- 172.21.15.57:0/412408301 wait complete. 2023-10-07T23:30:35.184 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 2 to deep-scrub 2023-10-07T23:30:35.194 INFO:tasks.ceph:Scrubbing osd.3 2023-10-07T23:30:35.194 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:35.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.303+0000 7fae86122700 1 Processor -- start 2023-10-07T23:30:35.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.303+0000 7fae86122700 1 -- start start 2023-10-07T23:30:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae80158c40 0x7fae80159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae8006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8006ac30 con 0x7fae800afd20 2023-10-07T23:30:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8015d9f0 con 0x7fae80158c40 2023-10-07T23:30:35.312 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae800ca530 con 0x7fae8015d580 2023-10-07T23:30:35.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fae800afd20 0x7fae8006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38902/0 (socket says 172.21.15.57:38902) 2023-10-07T23:30:35.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 -- 172.21.15.57:0/2943517016 learned_addr learned my addr 172.21.15.57:0/2943517016 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:35.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85120700 1 --2- 172.21.15.57:0/2943517016 >> v2:172.21.15.154:3300/0 conn(0x7fae80158c40 0x7fae80159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.315 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae8491f700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae8006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476555555 0 0) 0x7fae8006ac30 con 0x7fae800afd20 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae6c003570 con 0x7fae800afd20 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v1:172.21.15.57:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197736767 0 0) 0x7fae6c003570 con 0x7fae800afd20 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 msgr2=0x7fae8006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae8006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 >> v2:172.21.15.154:3300/0 conn(0x7fae80158c40 msgr2=0x7fae80159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.316 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 --2- 172.21.15.57:0/2943517016 >> v2:172.21.15.154:3300/0 conn(0x7fae80158c40 0x7fae80159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 --> v1:172.21.15.57:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8006ae60 con 0x7fae800afd20 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v1:172.21.15.57:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 476451635 0 0) 0x7fae780017f0 con 0x7fae800afd20 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 >> v1:172.21.15.57:6789/0 conn(0x7fae800afd20 legacy=0x7fae8006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c003a60 0x7fae6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae6c004d50 0x7fae6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae800ca530 con 0x7fae6c004d50 2023-10-07T23:30:35.317 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c00ba60 con 0x7fae6c003a60 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c00bba0 con 0x7fae8015d580 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae6c004d50 0x7fae6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85120700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c003a60 0x7fae6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae8491f700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 -- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 msgr2=0x7fae6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.318 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 -- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c003a60 msgr2=0x7fae6c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c003a60 0x7fae6c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 -- 172.21.15.57:0/2943517016 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c003570 con 0x7fae6c004d50 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae8491f700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae8015d580 0x7fae6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae6c004d50 0x7fae6c0051c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fae7800a970 tx=0x7fae800ca4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa15971f3695479d server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780070d0 con 0x7fae6c004d50 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae78007270 con 0x7fae6c004d50 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae73fff700 1 -- 172.21.15.57:0/2943517016 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78007630 con 0x7fae6c004d50 2023-10-07T23:30:35.319 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- 172.21.15.57:0/2943517016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae6c004d50 msgr2=0x7fae6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 --2- 172.21.15.57:0/2943517016 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae6c004d50 0x7fae6c0051c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fae7800a970 tx=0x7fae800ca4a0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae85921700 1 -- 172.21.15.57:0/2943517016 reap_dead start 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- 172.21.15.57:0/2943517016 shutdown_connections 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- 172.21.15.57:0/2943517016 >> 172.21.15.57:0/2943517016 conn(0x7fae80062ba0 msgr2=0x7fae80062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- 172.21.15.57:0/2943517016 shutdown_connections 2023-10-07T23:30:35.320 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.307+0000 7fae86122700 1 -- 172.21.15.57:0/2943517016 wait complete. 2023-10-07T23:30:35.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 Processor -- start 2023-10-07T23:30:35.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- start start 2023-10-07T23:30:35.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c004d50 0x7fae80160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.323 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 0x7fae801579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae80159010 0x7fae80157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8015a190 con 0x7fae80160c80 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae80158420 con 0x7fae6c004d50 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae80159580 con 0x7fae80159010 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 0x7fae801579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.325 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 0x7fae801579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40982/0 (socket says 172.21.15.57:40982) 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 -- 172.21.15.57:0/2515900836 learned_addr learned my addr 172.21.15.57:0/2515900836 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae85921700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae80159010 0x7fae80157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae85120700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c004d50 0x7fae80160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 -- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae80159010 msgr2=0x7fae80157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae80159010 0x7fae80157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 -- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c004d50 msgr2=0x7fae80160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.326 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae6c004d50 0x7fae80160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80159860 con 0x7fae80160c80 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae85921700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae80159010 0x7fae80157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae8491f700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 0x7fae801579a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fae7c010c70 tx=0x7fae7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00e9f0 con 0x7fae80160c80 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c00eb50 con 0x7fae80160c80 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c01c400 con 0x7fae80160c80 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae80159b40 con 0x7fae80160c80 2023-10-07T23:30:35.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae800c92b0 con 0x7fae80160c80 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.311+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae800711c0 con 0x7fae80160c80 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fae7c01c7f0 con 0x7fae80160c80 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fae64060550 0x7fae64062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fae7c07ca00 con 0x7fae80160c80 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7fae640663f0 0x7fae64068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae64068eb0 con 0x7fae640663f0 2023-10-07T23:30:35.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7c081240 con 0x7fae80160c80 2023-10-07T23:30:35.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae85120700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fae64060550 0x7fae64062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae85921700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7fae640663f0 0x7fae64068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae85921700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7fae640663f0 0x7fae64068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae85120700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fae64060550 0x7fae64062a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae74006c80 tx=0x7fae74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.315+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== osd.3 v2:172.21.15.57:6808/33394 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fae64068eb0 con 0x7fae640663f0 2023-10-07T23:30:35.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 --> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fae800c8e00 con 0x7fae640663f0 2023-10-07T23:30:35.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae717fa700 1 -- 172.21.15.57:0/2515900836 <== osd.3 v2:172.21.15.57:6808/33394 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fae800c8e00 con 0x7fae640663f0 2023-10-07T23:30:35.338 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:35.339 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:35.339 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:35.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7fae640663f0 msgr2=0x7fae64068800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:6808/33394,v1:172.21.15.57:6809/33394] conn(0x7fae640663f0 0x7fae64068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fae64060550 msgr2=0x7fae64062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fae64060550 0x7fae64062a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae74006c80 tx=0x7fae74008040 comp rx=0 tx=0).stop 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 msgr2=0x7fae801579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 --2- 172.21.15.57:0/2515900836 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fae80160c80 0x7fae801579a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fae7c010c70 tx=0x7fae7c008040 comp rx=0 tx=0).stop 2023-10-07T23:30:35.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae85921700 1 -- 172.21.15.57:0/2515900836 reap_dead start 2023-10-07T23:30:35.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 shutdown_connections 2023-10-07T23:30:35.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 >> 172.21.15.57:0/2515900836 conn(0x7fae80062ba0 msgr2=0x7fae800b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 shutdown_connections 2023-10-07T23:30:35.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.331+0000 7fae86122700 1 -- 172.21.15.57:0/2515900836 wait complete. 2023-10-07T23:30:35.356 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-07T23:30:35.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 Processor -- start 2023-10-07T23:30:35.468 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- start start 2023-10-07T23:30:35.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0b1c30 0x7ff1ec06de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.470 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff1ec0af010 0x7ff1ec15d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec0adc80 con 0x7ff1ec06da40 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0ad390 con 0x7ff1ec0af010 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0ad630 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f15bf700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7ff1ec06da40 0x7ff1ec15c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38906/0 (socket says 172.21.15.57:38906) 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7ff1ec0af010 0x7ff1ec15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.471 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f15bf700 1 -- 172.21.15.57:0/4202333476 learned_addr learned my addr 172.21.15.57:0/4202333476 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f0dbe700 1 --2- 172.21.15.57:0/4202333476 >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0b1c30 0x7ff1ec06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1e3fff700 1 -- 172.21.15.57:0/4202333476 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191445432 0 0) 0x7ff1ec0adc80 con 0x7ff1ec06da40 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1e3fff700 1 -- 172.21.15.57:0/4202333476 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d8003570 con 0x7ff1ec06da40 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/4202333476 >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0b1c30 msgr2=0x7ff1ec06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 --2- 172.21.15.57:0/4202333476 >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0b1c30 0x7ff1ec06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/4202333476 >> v1:172.21.15.57:6789/0 conn(0x7ff1ec06da40 legacy=0x7ff1ec15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/4202333476 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0ad910 con 0x7ff1ec0af010 2023-10-07T23:30:35.472 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f1dc0700 1 --2- 172.21.15.57:0/4202333476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0af010 0x7ff1ec15d080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06a760 tx=0x7ff1e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fb5fa7f09deeff server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1e3fff700 1 -- 172.21.15.57:0/4202333476 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4011630 con 0x7ff1ec0af010 2023-10-07T23:30:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1e3fff700 1 -- 172.21.15.57:0/4202333476 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40117b0 con 0x7ff1ec0af010 2023-10-07T23:30:35.473 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1e3fff700 1 -- 172.21.15.57:0/4202333476 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4011b70 con 0x7ff1ec0af010 2023-10-07T23:30:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- 172.21.15.57:0/4202333476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0af010 msgr2=0x7ff1ec15d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/4202333476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0af010 0x7ff1ec15d080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06a760 tx=0x7ff1e40091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- 172.21.15.57:0/4202333476 shutdown_connections 2023-10-07T23:30:35.474 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/4202333476 >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0b1c30 0x7ff1ec06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/4202333476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0af010 0x7ff1ec15d080 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- 172.21.15.57:0/4202333476 >> 172.21.15.57:0/4202333476 conn(0x7ff1ec06b710 msgr2=0x7ff1ec0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- 172.21.15.57:0/4202333476 shutdown_connections 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.463+0000 7ff1f25c1700 1 -- 172.21.15.57:0/4202333476 wait complete. 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 Processor -- start 2023-10-07T23:30:35.475 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 -- start start 2023-10-07T23:30:35.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ec06da40 0x7ff1ec0aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0af010 0x7ff1ec14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.477 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec14c700 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0a97a0 con 0x7ff1ec0af010 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f25c1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec15e080 con 0x7ff1ec06da40 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41000/0 (socket says 172.21.15.57:41000) 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/3897074393 learned_addr learned my addr 172.21.15.57:0/3897074393 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:35.478 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f0dbe700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0af010 0x7ff1ec14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f15bf700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ec06da40 0x7ff1ec0aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ec06da40 msgr2=0x7ff1ec0aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ec06da40 0x7ff1ec0aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0af010 msgr2=0x7ff1ec14bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0af010 0x7ff1ec14bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.467+0000 7ff1f1dc0700 1 -- 172.21.15.57:0/3897074393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4009420 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.479 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1f1dc0700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06bc50 tx=0x7ff1e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401f070 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e4006af0 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0a99f0 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4011b50 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.480 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0a9fb0 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec0711c0 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7ff1e4023070 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 0x7ff1d0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.481 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.471+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7ff1e407d910 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.475+0000 7ff1f15bf700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 0x7ff1d0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.482 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.475+0000 7ff1f15bf700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 0x7ff1d0062a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005c50 tx=0x7ff1e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.488 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.483+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff1e404e250 con 0x7ff1ec0b1c30 2023-10-07T23:30:35.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1ec0b20f0 con 0x7ff1d0060550 2023-10-07T23:30:35.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1e17fa700 1 -- 172.21.15.57:0/3897074393 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff1ec0b20f0 con 0x7ff1d0060550 2023-10-07T23:30:35.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 msgr2=0x7ff1d0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 0x7ff1d0062a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005c50 tx=0x7ff1e8009040 comp rx=0 tx=0).stop 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 msgr2=0x7ff1ec14c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec06bc50 tx=0x7ff1e4017000 comp rx=0 tx=0).stop 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 shutdown_connections 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1ec0b1c30 0x7ff1ec14c1c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7ff1d0060550 0x7ff1d0062a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0af010 0x7ff1ec14bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 --2- 172.21.15.57:0/3897074393 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff1ec06da40 0x7ff1ec0aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 >> 172.21.15.57:0/3897074393 conn(0x7ff1ec06b710 msgr2=0x7ff1ec0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 shutdown_connections 2023-10-07T23:30:35.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.647+0000 7ff1f25c1700 1 -- 172.21.15.57:0/3897074393 wait complete. 2023-10-07T23:30:35.657 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 3 to deep-scrub 2023-10-07T23:30:35.667 INFO:tasks.ceph:Scrubbing osd.4 2023-10-07T23:30:35.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:35.783 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.775+0000 7f07af59e700 1 Processor -- start 2023-10-07T23:30:35.784 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.775+0000 7f07af59e700 1 -- start start 2023-10-07T23:30:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b015d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.785 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b006a6f0 0x7f07b006c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b00ca530 con 0x7f07b0158c40 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b006ae60 con 0x7f07b006a6f0 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b006b140 con 0x7f07b0159010 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b006a6f0 0x7f07b006c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ae59c700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f07b0158c40 0x7f07b00afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38908/0 (socket says 172.21.15.57:38908) 2023-10-07T23:30:35.786 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f07b006a6f0 0x7f07b006c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41004/0 (socket says 172.21.15.57:41004) 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ae59c700 1 -- 172.21.15.57:0/2399099747 learned_addr learned my addr 172.21.15.57:0/2399099747 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07add9b700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 -- 172.21.15.57:0/2399099747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 msgr2=0x7f07b015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 -- 172.21.15.57:0/2399099747 >> v1:172.21.15.57:6789/0 conn(0x7f07b0158c40 legacy=0x7f07b00afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ad59a700 1 -- 172.21.15.57:0/2399099747 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724304723 0 0) 0x7f07b00ca530 con 0x7f07b0158c40 2023-10-07T23:30:35.787 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 -- 172.21.15.57:0/2399099747 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b006b420 con 0x7f07b006a6f0 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07aed9d700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b006a6f0 0x7f07b006c800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07a80098e0 tx=0x7f07a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab33e39b0a7ad89b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ad59a700 1 -- 172.21.15.57:0/2399099747 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800b640 con 0x7f07b006a6f0 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ad59a700 1 -- 172.21.15.57:0/2399099747 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a800b7a0 con 0x7f07b006a6f0 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07ad59a700 1 -- 172.21.15.57:0/2399099747 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a800bb40 con 0x7f07b006a6f0 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- 172.21.15.57:0/2399099747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b006a6f0 msgr2=0x7f07b006c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b006a6f0 0x7f07b006c800 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f07a80098e0 tx=0x7f07a80091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- 172.21.15.57:0/2399099747 shutdown_connections 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 --2- 172.21.15.57:0/2399099747 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b006a6f0 0x7f07b006c800 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- 172.21.15.57:0/2399099747 >> 172.21.15.57:0/2399099747 conn(0x7f07b0062ba0 msgr2=0x7f07b0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- 172.21.15.57:0/2399099747 shutdown_connections 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.779+0000 7f07af59e700 1 -- 172.21.15.57:0/2399099747 wait complete. 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 Processor -- start 2023-10-07T23:30:35.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 -- start start 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b006a6f0 0x7f07b00b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 0x7f07b00cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0159ae0 con 0x7f07b0158c40 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0154170 con 0x7f07b006a6f0 2023-10-07T23:30:35.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00b2f30 con 0x7f07b0159010 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 0x7f07b00cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07ae59c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b006a6f0 0x7f07b00b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 0x7f07b00cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41020/0 (socket says 172.21.15.57:41020) 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 -- 172.21.15.57:0/2240129820 learned_addr learned my addr 172.21.15.57:0/2240129820 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07aed9d700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 -- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 msgr2=0x7f07b00cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.794 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f07b0159010 0x7f07b00cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 -- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b006a6f0 msgr2=0x7f07b00b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b006a6f0 0x7f07b00b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009420 con 0x7f07b0158c40 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07ae59c700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07b006a6f0 0x7f07b00b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07add9b700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 0x7f07b00cd920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f07a000ec80 tx=0x7f07a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000c850 con 0x7f07b0158c40 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a000c9b0 con 0x7f07b0158c40 2023-10-07T23:30:35.795 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b0153e20 con 0x7f07b0158c40 2023-10-07T23:30:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.783+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a000cd50 con 0x7f07b0158c40 2023-10-07T23:30:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b00cea50 con 0x7f07b0158c40 2023-10-07T23:30:35.796 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07b00711c0 con 0x7f07b0158c40 2023-10-07T23:30:35.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f07a0013680 con 0x7f07b0158c40 2023-10-07T23:30:35.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0790060550 0x7f0790062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f07a007cb10 con 0x7f07b0158c40 2023-10-07T23:30:35.797 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f07900663f0 0x7f0790068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0790068eb0 con 0x7f07900663f0 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a007cee0 con 0x7f07b0158c40 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07ae59c700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0790060550 0x7f0790062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.787+0000 7f07aed9d700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f07900663f0 0x7f0790068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.791+0000 7f07ae59c700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0790060550 0x7f0790062a00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0798001560 tx=0x7f0798001a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.798 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.791+0000 7f07aed9d700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f07900663f0 0x7f0790068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.799 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.791+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== osd.4 v2:172.21.15.154:6808/33720 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0790068eb0 con 0x7f07900663f0 2023-10-07T23:30:35.816 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 --> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f07b00cf340 con 0x7f07900663f0 2023-10-07T23:30:35.817 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07a67fc700 1 -- 172.21.15.57:0/2240129820 <== osd.4 v2:172.21.15.154:6808/33720 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f07b00cf340 con 0x7f07900663f0 2023-10-07T23:30:35.817 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:35.817 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:35.818 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:35.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f07900663f0 msgr2=0x7f0790068800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6808/33720,v1:172.21.15.154:6809/33720] conn(0x7f07900663f0 0x7f0790068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0790060550 msgr2=0x7f0790062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f0790060550 0x7f0790062a00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0798001560 tx=0x7f0798001a20 comp rx=0 tx=0).stop 2023-10-07T23:30:35.819 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 msgr2=0x7f07b00cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 --2- 172.21.15.57:0/2240129820 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f07b0158c40 0x7f07b00cd920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f07a000ec80 tx=0x7f07a00091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07aed9d700 1 -- 172.21.15.57:0/2240129820 reap_dead start 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 shutdown_connections 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 >> 172.21.15.57:0/2240129820 conn(0x7f07b0062ba0 msgr2=0x7f07b00d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 shutdown_connections 2023-10-07T23:30:35.820 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.811+0000 7f07af59e700 1 -- 172.21.15.57:0/2240129820 wait complete. 2023-10-07T23:30:35.835 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-07T23:30:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.947+0000 7f7398ecc700 1 Processor -- start 2023-10-07T23:30:35.956 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.947+0000 7f7398ecc700 1 -- start start 2023-10-07T23:30:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 0x7f7394158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73940a88c0 0x7f739406b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73940a8d10 con 0x7f73940afd50 2023-10-07T23:30:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73940af290 con 0x7f73940a88c0 2023-10-07T23:30:35.957 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739406c030 con 0x7f7394158bd0 2023-10-07T23:30:35.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f73940a88c0 0x7f739406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7393fff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f73940afd50 0x7f73940b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38920/0 (socket says 172.21.15.57:38920) 2023-10-07T23:30:35.958 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7393fff700 1 -- 172.21.15.57:0/2191579165 learned_addr learned my addr 172.21.15.57:0/2191579165 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:35.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f73937fe700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 0x7f7394158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 -- 172.21.15.57:0/2191579165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 msgr2=0x7f7394158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:35.959 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 0x7f7394158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 -- 172.21.15.57:0/2191579165 >> v1:172.21.15.57:6789/0 conn(0x7f73940afd50 legacy=0x7f73940b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7392ffd700 1 -- 172.21.15.57:0/2191579165 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18540773 0 0) 0x7f73940a8d10 con 0x7f73940afd50 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 -- 172.21.15.57:0/2191579165 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73940af550 con 0x7f73940a88c0 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f73937fe700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 0x7f7394158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f738bfff700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73940a88c0 0x7f739406b800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f737c0098e0 tx=0x7f737c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f0627d81e413b server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7392ffd700 1 -- 172.21.15.57:0/2191579165 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c009e80 con 0x7f73940a88c0 2023-10-07T23:30:35.960 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7392ffd700 1 -- 172.21.15.57:0/2191579165 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00b860 con 0x7f73940a88c0 2023-10-07T23:30:35.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7392ffd700 1 -- 172.21.15.57:0/2191579165 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00bbe0 con 0x7f73940a88c0 2023-10-07T23:30:35.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 -- 172.21.15.57:0/2191579165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73940a88c0 msgr2=0x7f739406b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.961 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.951+0000 7f7398ecc700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73940a88c0 0x7f739406b800 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f737c0098e0 tx=0x7f737c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- 172.21.15.57:0/2191579165 shutdown_connections 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73940a88c0 0x7f739406b800 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 --2- 172.21.15.57:0/2191579165 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7394158bd0 0x7f7394158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- 172.21.15.57:0/2191579165 >> 172.21.15.57:0/2191579165 conn(0x7f73940cce10 msgr2=0x7f73940add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- 172.21.15.57:0/2191579165 shutdown_connections 2023-10-07T23:30:35.962 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- 172.21.15.57:0/2191579165 wait complete. 2023-10-07T23:30:35.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 Processor -- start 2023-10-07T23:30:35.963 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- start start 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73940a88c0 0x7f7394141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73940afd50 0x7f7394144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7394144480 con 0x7f7394158bd0 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7394141e90 con 0x7f73940afd50 2023-10-07T23:30:35.965 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.955+0000 7f7398ecc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7394143870 con 0x7f73940a88c0 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41050/0 (socket says 172.21.15.57:41050) 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 -- 172.21.15.57:0/3236842783 learned_addr learned my addr 172.21.15.57:0/3236842783 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738bfff700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73940afd50 0x7f7394144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f73937fe700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73940a88c0 0x7f7394141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 -- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73940a88c0 msgr2=0x7f7394141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.966 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73940a88c0 0x7f7394141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 -- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73940afd50 msgr2=0x7f7394144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73940afd50 0x7f7394144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 -- 172.21.15.57:0/3236842783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c009420 con 0x7f7394158bd0 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7393fff700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f738c00cc90 tx=0x7f738c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00a8b0 con 0x7f7394158bd0 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c00aa10 con 0x7f7394158bd0 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7394143b50 con 0x7f7394158bd0 2023-10-07T23:30:35.967 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00add0 con 0x7f7394158bd0 2023-10-07T23:30:35.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7394144110 con 0x7f7394158bd0 2023-10-07T23:30:35.968 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73941445f0 con 0x7f7394158bd0 2023-10-07T23:30:35.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f738c007300 con 0x7f7394158bd0 2023-10-07T23:30:35.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.959+0000 7f738b7fe700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 0x7f7374062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:35.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.963+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f738c07c860 con 0x7f7394158bd0 2023-10-07T23:30:35.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.963+0000 7f73937fe700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 0x7f7374062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:35.969 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.963+0000 7f73937fe700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 0x7f7374062a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7384006f00 tx=0x7f7384008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:35.977 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:35.971+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f738c04d120 con 0x7f7394158bd0 2023-10-07T23:30:36.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.135+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f73940aea50 con 0x7f7374060580 2023-10-07T23:30:36.146 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f738b7fe700 1 -- 172.21.15.57:0/3236842783 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f73940aea50 con 0x7f7374060580 2023-10-07T23:30:36.147 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 msgr2=0x7f7374062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 0x7f7374062a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7384006f00 tx=0x7f7384008040 comp rx=0 tx=0).stop 2023-10-07T23:30:36.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 msgr2=0x7f73941576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f738c00cc90 tx=0x7f738c001a80 comp rx=0 tx=0).stop 2023-10-07T23:30:36.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 shutdown_connections 2023-10-07T23:30:36.148 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7394158bd0 0x7f73941576d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f7374060580 0x7f7374062a30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73940afd50 0x7f7394144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 --2- 172.21.15.57:0/3236842783 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f73940a88c0 0x7f7394141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 >> 172.21.15.57:0/3236842783 conn(0x7f73940cce10 msgr2=0x7f7394156650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 shutdown_connections 2023-10-07T23:30:36.149 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.139+0000 7f7398ecc700 1 -- 172.21.15.57:0/3236842783 wait complete. 2023-10-07T23:30:36.150 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 4 to deep-scrub 2023-10-07T23:30:36.160 INFO:tasks.ceph:Scrubbing osd.5 2023-10-07T23:30:36.160 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.267+0000 7f13fbc0a700 1 Processor -- start 2023-10-07T23:30:36.275 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.267+0000 7f13fbc0a700 1 -- start start 2023-10-07T23:30:36.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f13f406cf10 0x7f13f40aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.276 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f406a900 con 0x7f13f40ca600 2023-10-07T23:30:36.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f406a600 con 0x7f13f4159030 2023-10-07T23:30:36.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40b0450 con 0x7f13f406cf10 2023-10-07T23:30:36.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f91a5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f13f406cf10 0x7f13f40aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fa1a7700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f13f40ca600 0x7f13f40caa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38936/0 (socket says 172.21.15.57:38936) 2023-10-07T23:30:36.277 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fa1a7700 1 -- 172.21.15.57:0/1015726114 learned_addr learned my addr 172.21.15.57:0/1015726114 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:36.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f89a4700 1 -- 172.21.15.57:0/1015726114 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538068097 0 0) 0x7f13f406a900 con 0x7f13f40ca600 2023-10-07T23:30:36.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f89a4700 1 -- 172.21.15.57:0/1015726114 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e0003570 con 0x7f13f40ca600 2023-10-07T23:30:36.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f91a5700 1 -- 172.21.15.57:0/1015726114 >> v1:172.21.15.154:6789/0 conn(0x7f13f4159030 legacy=0x7f13f406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f91a5700 1 -- 172.21.15.57:0/1015726114 >> v1:172.21.15.57:6789/0 conn(0x7f13f40ca600 legacy=0x7f13f40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.279 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f91a5700 1 -- 172.21.15.57:0/1015726114 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40cb350 con 0x7f13f406cf10 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f91a5700 1 --2- 172.21.15.57:0/1015726114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f406cf10 0x7f13f40aff10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098e0 tx=0x7f13f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3d8d18c9011072 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f89a4700 1 -- 172.21.15.57:0/1015726114 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000b640 con 0x7f13f406cf10 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f89a4700 1 -- 172.21.15.57:0/1015726114 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000b7a0 con 0x7f13f406cf10 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13f89a4700 1 -- 172.21.15.57:0/1015726114 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000bb20 con 0x7f13f406cf10 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- 172.21.15.57:0/1015726114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f406cf10 msgr2=0x7f13f40aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 --2- 172.21.15.57:0/1015726114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f406cf10 0x7f13f40aff10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f13f00098e0 tx=0x7f13f00091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.280 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- 172.21.15.57:0/1015726114 shutdown_connections 2023-10-07T23:30:36.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 --2- 172.21.15.57:0/1015726114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f406cf10 0x7f13f40aff10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- 172.21.15.57:0/1015726114 >> 172.21.15.57:0/1015726114 conn(0x7f13f4068650 msgr2=0x7f13f4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- 172.21.15.57:0/1015726114 shutdown_connections 2023-10-07T23:30:36.281 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.271+0000 7f13fbc0a700 1 -- 172.21.15.57:0/1015726114 wait complete. 2023-10-07T23:30:36.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.275+0000 7f13fbc0a700 1 Processor -- start 2023-10-07T23:30:36.282 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.275+0000 7f13fbc0a700 1 -- start start 2023-10-07T23:30:36.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13f406cf10 0x7f13f4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.284 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13f40ca600 0x7f13f40c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 0x7f13f4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40b1240 con 0x7f13f4159030 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40b0630 con 0x7f13f40ca600 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40b0910 con 0x7f13f406cf10 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 0x7f13f4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.285 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13f99a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13f406cf10 0x7f13f4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 0x7f13f4151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41066/0 (socket says 172.21.15.57:41066) 2023-10-07T23:30:36.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13f99a6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13f406cf10 0x7f13f4159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.57:41394/0 (socket says 172.21.15.57:41394) 2023-10-07T23:30:36.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 -- 172.21.15.57:0/112537896 learned_addr learned my addr 172.21.15.57:0/112537896 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:36.286 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13f91a5700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13f40ca600 0x7f13f40c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.287 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 -- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13f406cf10 msgr2=0x7f13f4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f13f406cf10 0x7f13f4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 -- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13f40ca600 msgr2=0x7f13f40c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13f40ca600 0x7f13f40c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009420 con 0x7f13f4159030 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fa1a7700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 0x7f13f4151c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001460 tx=0x7f13ec001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00a890 con 0x7f13f4159030 2023-10-07T23:30:36.288 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00a9f0 con 0x7f13f4159030 2023-10-07T23:30:36.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f40b0bf0 con 0x7f13f4159030 2023-10-07T23:30:36.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00ad70 con 0x7f13f4159030 2023-10-07T23:30:36.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f40b16d0 con 0x7f13f4159030 2023-10-07T23:30:36.289 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.279+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13f40b5e80 con 0x7f13f4159030 2023-10-07T23:30:36.290 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f13ec007300 con 0x7f13f4159030 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13d8060580 0x7f13d8062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f13ec07c930 con 0x7f13f4159030 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f13d8066420 0x7f13d8068830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13d8068ee0 con 0x7f13d8066420 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13ec081240 con 0x7f13f4159030 2023-10-07T23:30:36.291 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13f99a6700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13d8060580 0x7f13d8062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13f91a5700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f13d8066420 0x7f13d8068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13f99a6700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13d8060580 0x7f13d8062a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13e4003aa0 tx=0x7f13e4000bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.283+0000 7f13f91a5700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f13d8066420 0x7f13d8068830 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.292 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.287+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== osd.5 v2:172.21.15.154:6824/33759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f13d8068ee0 con 0x7f13d8066420 2023-10-07T23:30:36.310 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.303+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 --> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13f40b2370 con 0x7f13d8066420 2023-10-07T23:30:36.311 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.303+0000 7f13e9ffb700 1 -- 172.21.15.57:0/112537896 <== osd.5 v2:172.21.15.154:6824/33759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f13f40b2370 con 0x7f13d8066420 2023-10-07T23:30:36.311 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:36.312 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:36.312 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.303+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f13d8066420 msgr2=0x7f13d8068830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6824/33759,v1:172.21.15.154:6825/33759] conn(0x7f13d8066420 0x7f13d8068830 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13d8060580 msgr2=0x7f13d8062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f13d8060580 0x7f13d8062a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13e4003aa0 tx=0x7f13e4000bc0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.313 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 msgr2=0x7f13f4151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 --2- 172.21.15.57:0/112537896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f13f4159030 0x7f13f4151c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f13ec001460 tx=0x7f13ec001ae0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fa1a7700 1 -- 172.21.15.57:0/112537896 reap_dead start 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 shutdown_connections 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 >> 172.21.15.57:0/112537896 conn(0x7f13f4068650 msgr2=0x7f13f4152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 shutdown_connections 2023-10-07T23:30:36.314 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.307+0000 7f13fbc0a700 1 -- 172.21.15.57:0/112537896 wait complete. 2023-10-07T23:30:36.325 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-07T23:30:36.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 Processor -- start 2023-10-07T23:30:36.437 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 -- start start 2023-10-07T23:30:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 0x7fac8406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8406bbf0 con 0x7fac84161a70 2023-10-07T23:30:36.438 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8406b020 con 0x7fac8415d2a0 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac83fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840c9010 con 0x7fac8415ced0 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac827fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fac84161a70 0x7fac840c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:38952/0 (socket says 172.21.15.57:38952) 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac827fc700 1 -- 172.21.15.57:0/4168147743 learned_addr learned my addr 172.21.15.57:0/4168147743 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac82ffd700 1 --2- 172.21.15.57:0/4168147743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 0x7fac8406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353186815 0 0) 0x7fac8406bbf0 con 0x7fac84161a70 2023-10-07T23:30:36.439 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac64003570 con 0x7fac84161a70 2023-10-07T23:30:36.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957739416 0 0) 0x7fac8406b020 con 0x7fac8415d2a0 2023-10-07T23:30:36.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac82ffd700 1 -- 172.21.15.57:0/4168147743 >> v1:172.21.15.154:6789/0 conn(0x7fac8415d2a0 legacy=0x7fac840c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac82ffd700 1 -- 172.21.15.57:0/4168147743 >> v1:172.21.15.57:6789/0 conn(0x7fac84161a70 legacy=0x7fac840c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.440 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac82ffd700 1 -- 172.21.15.57:0/4168147743 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8406b2c0 con 0x7fac8415ced0 2023-10-07T23:30:36.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac82ffd700 1 --2- 172.21.15.57:0/4168147743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 0x7fac8406c080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009910 tx=0x7fac6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a24174ca65c96bb server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.441 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00b680 con 0x7fac8415ced0 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c00b7e0 con 0x7fac8415ced0 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.431+0000 7fac81ffb700 1 -- 172.21.15.57:0/4168147743 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00bba0 con 0x7fac8415ced0 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- 172.21.15.57:0/4168147743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 msgr2=0x7fac8406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 --2- 172.21.15.57:0/4168147743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 0x7fac8406c080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009910 tx=0x7fac6c0091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- 172.21.15.57:0/4168147743 shutdown_connections 2023-10-07T23:30:36.442 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 --2- 172.21.15.57:0/4168147743 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac8415ced0 0x7fac8406c080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- 172.21.15.57:0/4168147743 >> 172.21.15.57:0/4168147743 conn(0x7fac840670e0 msgr2=0x7fac840674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- 172.21.15.57:0/4168147743 shutdown_connections 2023-10-07T23:30:36.443 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- 172.21.15.57:0/4168147743 wait complete. 2023-10-07T23:30:36.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 Processor -- start 2023-10-07T23:30:36.445 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- start start 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 0x7fac840ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac84161a70 0x7fac84152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8415e120 con 0x7fac8415ced0 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840af090 con 0x7fac8415d2a0 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac83fff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840af370 con 0x7fac84161a70 2023-10-07T23:30:36.446 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac82ffd700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41080/0 (socket says 172.21.15.57:41080) 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac82ffd700 1 -- 172.21.15.57:0/512715349 learned_addr learned my addr 172.21.15.57:0/512715349 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac827fc700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 0x7fac840ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.435+0000 7fac837fe700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac84161a70 0x7fac84152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 -- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac84161a70 msgr2=0x7fac84152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac84161a70 0x7fac84152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.447 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 -- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 msgr2=0x7fac840ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 0x7fac840ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 -- 172.21.15.57:0/512715349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c009420 con 0x7fac8415ced0 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac827fc700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 0x7fac840ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac82ffd700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac8406ec20 tx=0x7fac6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00bb60 con 0x7fac8415ced0 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c027450 con 0x7fac8415ced0 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0277f0 con 0x7fac8415ced0 2023-10-07T23:30:36.448 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac840af650 con 0x7fac8415ced0 2023-10-07T23:30:36.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac840aeda0 con 0x7fac8415ced0 2023-10-07T23:30:36.449 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8415def0 con 0x7fac8415ced0 2023-10-07T23:30:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fac6c01a430 con 0x7fac8415ced0 2023-10-07T23:30:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.439+0000 7fac7affd700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 0x7fac68062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.443+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fac6c07e4b0 con 0x7fac8415ced0 2023-10-07T23:30:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.443+0000 7fac827fc700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 0x7fac68062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.450 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.443+0000 7fac827fc700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 0x7fac68062a00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fac8415dce0 tx=0x7fac7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.454 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.447+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fac8415def0 con 0x7fac8415ced0 2023-10-07T23:30:36.614 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.607+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fac8415e260 con 0x7fac68060550 2023-10-07T23:30:36.615 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.607+0000 7fac7affd700 1 -- 172.21.15.57:0/512715349 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fac7c00ec20 con 0x7fac68060550 2023-10-07T23:30:36.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 msgr2=0x7fac68062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 0x7fac68062a00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fac8415dce0 tx=0x7fac7c009040 comp rx=0 tx=0).stop 2023-10-07T23:30:36.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 msgr2=0x7fac841592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.619 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac8406ec20 tx=0x7fac6c017000 comp rx=0 tx=0).stop 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 shutdown_connections 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fac84161a70 0x7fac84152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fac68060550 0x7fac68062a00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac8415d2a0 0x7fac840ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 --2- 172.21.15.57:0/512715349 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fac8415ced0 0x7fac841592c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 >> 172.21.15.57:0/512715349 conn(0x7fac840670e0 msgr2=0x7fac8415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.620 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 shutdown_connections 2023-10-07T23:30:36.621 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.611+0000 7fac83fff700 1 -- 172.21.15.57:0/512715349 wait complete. 2023-10-07T23:30:36.621 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 5 to deep-scrub 2023-10-07T23:30:36.634 INFO:tasks.ceph:Scrubbing osd.6 2023-10-07T23:30:36.635 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:36.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 Processor -- start 2023-10-07T23:30:36.754 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 -- start start 2023-10-07T23:30:36.756 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 0x7f05a0159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a00ca530 con 0x7f05a015d580 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a006ae60 con 0x7f05a006a6f0 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a015d9f0 con 0x7f05a0158c40 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059e7fc700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f05a015d580 0x7f05a00afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:50108/0 (socket says 172.21.15.57:50108) 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.747+0000 7f059e7fc700 1 -- 172.21.15.57:0/3242116312 learned_addr learned my addr 172.21.15.57:0/3242116312 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:36.757 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059effd700 1 --2- 172.21.15.57:0/3242116312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 0x7f05a0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801283033 0 0) 0x7f05a00ca530 con 0x7f05a015d580 2023-10-07T23:30:36.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0588003570 con 0x7f05a015d580 2023-10-07T23:30:36.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441018700 0 0) 0x7f05a006ae60 con 0x7f05a006a6f0 2023-10-07T23:30:36.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a00ca530 con 0x7f05a006a6f0 2023-10-07T23:30:36.758 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059effd700 1 -- 172.21.15.57:0/3242116312 >> v1:172.21.15.154:6789/0 conn(0x7f05a006a6f0 legacy=0x7f05a006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059effd700 1 -- 172.21.15.57:0/3242116312 >> v1:172.21.15.57:6789/0 conn(0x7f05a015d580 legacy=0x7f05a00afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.761 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059effd700 1 -- 172.21.15.57:0/3242116312 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a006b140 con 0x7f05a0158c40 2023-10-07T23:30:36.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059effd700 1 --2- 172.21.15.57:0/3242116312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 0x7f05a0159010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0594009910 tx=0x7f05940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dce41ad793afbd82 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.764 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011630 con 0x7f05a0158c40 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594011790 con 0x7f05a0158c40 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059dffb700 1 -- 172.21.15.57:0/3242116312 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594011b30 con 0x7f05a0158c40 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 -- 172.21.15.57:0/3242116312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 msgr2=0x7f05a0159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 --2- 172.21.15.57:0/3242116312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 0x7f05a0159010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0594009910 tx=0x7f05940091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 -- 172.21.15.57:0/3242116312 shutdown_connections 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 --2- 172.21.15.57:0/3242116312 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a0158c40 0x7f05a0159010 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.765 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 -- 172.21.15.57:0/3242116312 >> 172.21.15.57:0/3242116312 conn(0x7f05a0062ba0 msgr2=0x7f05a0062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 -- 172.21.15.57:0/3242116312 shutdown_connections 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.751+0000 7f059ffff700 1 -- 172.21.15.57:0/3242116312 wait complete. 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 Processor -- start 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 -- start start 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05a006a6f0 0x7f05a00b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05a0158c40 0x7f05a00b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a015d580 0x7f05a015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.766 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00b3a40 con 0x7f05a0158c40 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00b2e30 con 0x7f05a006a6f0 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00b3110 con 0x7f05a015d580 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05a0158c40 0x7f05a00b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059e7fc700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05a0158c40 0x7f05a00b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46956/0 (socket says 172.21.15.57:46956) 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059e7fc700 1 -- 172.21.15.57:0/2696631525 learned_addr learned my addr 172.21.15.57:0/2696631525 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a015d580 0x7f05a015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.767 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059effd700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05a006a6f0 0x7f05a00b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 -- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05a006a6f0 msgr2=0x7f05a00b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05a006a6f0 0x7f05a00b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 -- 172.21.15.57:0/2696631525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05a0158c40 msgr2=0x7f05a00b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05a0158c40 0x7f05a00b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594009420 con 0x7f05a015d580 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059f7fe700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a015d580 0x7f05a015a580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f059800c8e0 tx=0x7f05980026f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598012630 con 0x7f05a015d580 2023-10-07T23:30:36.768 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598012790 con 0x7f05a015d580 2023-10-07T23:30:36.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a00b33f0 con 0x7f05a015d580 2023-10-07T23:30:36.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.755+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598012b50 con 0x7f05a015d580 2023-10-07T23:30:36.769 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.759+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a015aac0 con 0x7f05a015d580 2023-10-07T23:30:36.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.759+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05a00711c0 con 0x7f05a015d580 2023-10-07T23:30:36.770 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.759+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f0598024430 con 0x7f05a015d580 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f059807cd10 con 0x7f05a015d580 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f059effd700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f058c0663f0 0x7f058c068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f058c068eb0 con 0x7f058c0663f0 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== mon.2 v2:172.21.15.154:3301/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f059807d000 con 0x7f05a015d580 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f059e7fc700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f058c0663f0 0x7f058c068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.771 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f059effd700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0594009860 tx=0x7f0594017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f059e7fc700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f058c0663f0 0x7f058c068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.772 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.763+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== osd.6 v2:172.21.15.154:6800/33721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f058c068eb0 con 0x7f058c0663f0 2023-10-07T23:30:36.789 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 --> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f05a00b3250 con 0x7f058c0663f0 2023-10-07T23:30:36.790 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f0586ffd700 1 -- 172.21.15.57:0/2696631525 <== osd.6 v2:172.21.15.154:6800/33721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f05a00b3250 con 0x7f058c0663f0 2023-10-07T23:30:36.790 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:36.790 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:36.790 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:36.791 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f058c0663f0 msgr2=0x7f058c068800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6800/33721,v1:172.21.15.154:6801/33721] conn(0x7f058c0663f0 0x7f058c068800 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 msgr2=0x7f058c062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f058c060550 0x7f058c062a00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0594009860 tx=0x7f0594017000 comp rx=0 tx=0).stop 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a015d580 msgr2=0x7f05a015a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 --2- 172.21.15.57:0/2696631525 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f05a015d580 0x7f05a015a580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f059800c8e0 tx=0x7f05980026f0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059f7fe700 1 -- 172.21.15.57:0/2696631525 reap_dead start 2023-10-07T23:30:36.792 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 shutdown_connections 2023-10-07T23:30:36.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 >> 172.21.15.57:0/2696631525 conn(0x7f05a0062ba0 msgr2=0x7f05a00ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 shutdown_connections 2023-10-07T23:30:36.793 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.783+0000 7f059ffff700 1 -- 172.21.15.57:0/2696631525 wait complete. 2023-10-07T23:30:36.803 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-07T23:30:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.907+0000 7f87eecac700 1 Processor -- start 2023-10-07T23:30:36.915 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.907+0000 7f87eecac700 1 -- start start 2023-10-07T23:30:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f87e8159230 0x7f87e80aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.917 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80caba0 con 0x7f87e80b04a0 2023-10-07T23:30:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e806b2d0 con 0x7f87e8158e60 2023-10-07T23:30:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e806bb20 con 0x7f87e8159230 2023-10-07T23:30:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87ed249700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f87e80b04a0 0x7f87e806c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:50110/0 (socket says 172.21.15.57:50110) 2023-10-07T23:30:36.918 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87ed249700 1 -- 172.21.15.57:0/3797010107 learned_addr learned my addr 172.21.15.57:0/3797010107 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e7fff700 1 --2- 172.21.15.57:0/3797010107 >> v2:172.21.15.57:3300/0 conn(0x7f87e8159230 0x7f87e80aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803649242 0 0) 0x7f87e80caba0 con 0x7f87e80b04a0 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87d4003570 con 0x7f87e80b04a0 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e7fff700 1 -- 172.21.15.57:0/3797010107 >> v1:172.21.15.154:6789/0 conn(0x7f87e8158e60 legacy=0x7f87e806abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218857478 0 0) 0x7f87e806b2d0 con 0x7f87e8158e60 2023-10-07T23:30:36.919 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e7fff700 1 -- 172.21.15.57:0/3797010107 >> v1:172.21.15.57:6789/0 conn(0x7f87e80b04a0 legacy=0x7f87e806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.920 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e7fff700 1 -- 172.21.15.57:0/3797010107 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e806b560 con 0x7f87e8159230 2023-10-07T23:30:36.921 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e7fff700 1 --2- 172.21.15.57:0/3797010107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e80aff60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098e0 tx=0x7f87d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6cbc4879c95168 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b640 con 0x7f87e8159230 2023-10-07T23:30:36.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800b7a0 con 0x7f87e8159230 2023-10-07T23:30:36.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87e77fe700 1 -- 172.21.15.57:0/3797010107 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800bb40 con 0x7f87e8159230 2023-10-07T23:30:36.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- 172.21.15.57:0/3797010107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 msgr2=0x7f87e80aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.922 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 --2- 172.21.15.57:0/3797010107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e80aff60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098e0 tx=0x7f87d80091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- 172.21.15.57:0/3797010107 shutdown_connections 2023-10-07T23:30:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 --2- 172.21.15.57:0/3797010107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e80aff60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- 172.21.15.57:0/3797010107 >> 172.21.15.57:0/3797010107 conn(0x7f87e8068740 msgr2=0x7f87e8068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:36.923 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- 172.21.15.57:0/3797010107 shutdown_connections 2023-10-07T23:30:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.911+0000 7f87eecac700 1 -- 172.21.15.57:0/3797010107 wait complete. 2023-10-07T23:30:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 Processor -- start 2023-10-07T23:30:36.924 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 -- start start 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80b04a0 0x7f87e815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87e8158e60 0x7f87e80d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80c6ff0 con 0x7f87e8159230 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e815c870 con 0x7f87e80b04a0 2023-10-07T23:30:36.927 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eecac700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8157940 con 0x7f87e8158e60 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46980/0 (socket says 172.21.15.57:46980) 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 -- 172.21.15.57:0/4269831722 learned_addr learned my addr 172.21.15.57:0/4269831722 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87e7fff700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87e8158e60 0x7f87e80d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87eca48700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80b04a0 0x7f87e815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 -- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87e8158e60 msgr2=0x7f87e80d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87e8158e60 0x7f87e80d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.928 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 -- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80b04a0 msgr2=0x7f87e815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:36.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80b04a0 0x7f87e815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:36.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.915+0000 7f87ed249700 1 -- 172.21.15.57:0/4269831722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009420 con 0x7f87e8159230 2023-10-07T23:30:36.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87ed249700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f87e000cca0 tx=0x7f87e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.929 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e000a8d0 con 0x7f87e8159230 2023-10-07T23:30:36.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000aa30 con 0x7f87e8159230 2023-10-07T23:30:36.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e815ad50 con 0x7f87e8159230 2023-10-07T23:30:36.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e000adb0 con 0x7f87e8159230 2023-10-07T23:30:36.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e815b310 con 0x7f87e8159230 2023-10-07T23:30:36.930 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e80c71a0 con 0x7f87e8159230 2023-10-07T23:30:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.919+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f87e0007300 con 0x7f87e8159230 2023-10-07T23:30:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.923+0000 7f87e4ff9700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 0x7f87c8062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.923+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f87e007c940 con 0x7f87e8159230 2023-10-07T23:30:36.931 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.923+0000 7f87eca48700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 0x7f87c8062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:36.932 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.923+0000 7f87eca48700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 0x7f87c8062a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007fd0 tx=0x7f87d0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:36.934 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:36.927+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f87e004d160 con 0x7f87e8159230 2023-10-07T23:30:37.098 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.091+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e80647b0 con 0x7f87c8060550 2023-10-07T23:30:37.099 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.091+0000 7f87e4ff9700 1 -- 172.21.15.57:0/4269831722 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f87e80647b0 con 0x7f87c8060550 2023-10-07T23:30:37.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 msgr2=0x7f87c8062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.101 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 0x7f87c8062a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007fd0 tx=0x7f87d0007d30 comp rx=0 tx=0).stop 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 msgr2=0x7f87e814bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f87e000cca0 tx=0x7f87e0001ae0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 shutdown_connections 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f87e8159230 0x7f87e814bf50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f87e8158e60 0x7f87e80d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f87c8060550 0x7f87c8062a00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.102 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 --2- 172.21.15.57:0/4269831722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80b04a0 0x7f87e815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 >> 172.21.15.57:0/4269831722 conn(0x7f87e8068740 msgr2=0x7f87e815cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 shutdown_connections 2023-10-07T23:30:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.095+0000 7f87eecac700 1 -- 172.21.15.57:0/4269831722 wait complete. 2023-10-07T23:30:37.103 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 6 to deep-scrub 2023-10-07T23:30:37.113 INFO:tasks.ceph:Scrubbing osd.7 2023-10-07T23:30:37.114 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-07T23:30:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.219+0000 7fb0dd20c700 1 Processor -- start 2023-10-07T23:30:37.227 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.219+0000 7fb0dd20c700 1 -- start start 2023-10-07T23:30:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb0d8159080 0x7fb0d815d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb0d8159450 0x7fb0d806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d806d180 con 0x7fb0d806cd40 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d80cb910 con 0x7fb0d8159450 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d80ca590 con 0x7fb0d8159080 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb0d8159450 0x7fb0d806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fb0d8159450 0x7fb0d806c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46984/0 (socket says 172.21.15.57:46984) 2023-10-07T23:30:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 -- 172.21.15.57:0/4139101811 learned_addr learned my addr 172.21.15.57:0/4139101811 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:37.230 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d6d9d700 1 --2- 172.21.15.57:0/4139101811 >> v2:172.21.15.154:3300/0 conn(0x7fb0d8159080 0x7fb0d815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 -- 172.21.15.57:0/4139101811 >> v2:172.21.15.154:3300/0 conn(0x7fb0d8159080 msgr2=0x7fb0d815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 --2- 172.21.15.57:0/4139101811 >> v2:172.21.15.154:3300/0 conn(0x7fb0d8159080 0x7fb0d815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 -- 172.21.15.57:0/4139101811 >> v1:172.21.15.154:6789/0 conn(0x7fb0d806cd40 legacy=0x7fb0d80cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.231 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 -- 172.21.15.57:0/4139101811 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d80ca870 con 0x7fb0d8159450 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d659c700 1 --2- 172.21.15.57:0/4139101811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d8159450 0x7fb0d806c800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80644a0 tx=0x7fb0c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2822aceb58bd614 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d5d9b700 1 -- 172.21.15.57:0/4139101811 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800b860 con 0x7fb0d8159450 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d5d9b700 1 -- 172.21.15.57:0/4139101811 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c800b9c0 con 0x7fb0d8159450 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0d5d9b700 1 -- 172.21.15.57:0/4139101811 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800bd60 con 0x7fb0d8159450 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- 172.21.15.57:0/4139101811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d8159450 msgr2=0x7fb0d806c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/4139101811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d8159450 0x7fb0d806c800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80644a0 tx=0x7fb0c8009250 comp rx=0 tx=0).stop 2023-10-07T23:30:37.232 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- 172.21.15.57:0/4139101811 shutdown_connections 2023-10-07T23:30:37.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/4139101811 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d8159450 0x7fb0d806c800 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/4139101811 >> v2:172.21.15.154:3300/0 conn(0x7fb0d8159080 0x7fb0d815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- 172.21.15.57:0/4139101811 >> 172.21.15.57:0/4139101811 conn(0x7fb0d8062ba0 msgr2=0x7fb0d8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- 172.21.15.57:0/4139101811 shutdown_connections 2023-10-07T23:30:37.233 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 -- 172.21.15.57:0/4139101811 wait complete. 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.223+0000 7fb0dd20c700 1 Processor -- start 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- start start 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 0x7fb0d80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d8159080 0x7fb0d8169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d8159450 0x7fb0d816a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d816b0c0 con 0x7fb0d806cd40 2023-10-07T23:30:37.236 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80b40e0 con 0x7fb0d8159450 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d80c9460 con 0x7fb0d8159080 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 0x7fb0d80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 0x7fb0d80c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46988/0 (socket says 172.21.15.57:46988) 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 -- 172.21.15.57:0/2430633578 learned_addr learned my addr 172.21.15.57:0/2430633578 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d659c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d8159080 0x7fb0d8169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.237 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d759e700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d8159450 0x7fb0d816a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 -- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d8159080 msgr2=0x7fb0d8169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fb0d8159080 0x7fb0d8169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 -- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d8159450 msgr2=0x7fb0d816a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0d8159450 0x7fb0d816a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80094f0 con 0x7fb0d806cd40 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0d6d9d700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 0x7fb0d80c9c90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d890 tx=0x7fb0c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.238 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40125f0 con 0x7fb0d806cd40 2023-10-07T23:30:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4012750 con 0x7fb0d806cd40 2023-10-07T23:30:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d816a6f0 con 0x7fb0d806cd40 2023-10-07T23:30:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4012ad0 con 0x7fb0d806cd40 2023-10-07T23:30:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d816acb0 con 0x7fb0d806cd40 2023-10-07T23:30:37.239 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.227+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0d80647b0 con 0x7fb0d806cd40 2023-10-07T23:30:37.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fb0c401e460 con 0x7fb0d806cd40 2023-10-07T23:30:37.240 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb0c0060580 0x7fb0c0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fb0c407d580 con 0x7fb0d806cd40 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7fb0c0066420 0x7fb0c0068830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0c0068ee0 con 0x7fb0c0066420 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c407d950 con 0x7fb0d806cd40 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0d659c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb0c0060580 0x7fb0c0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0d759e700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7fb0c0066420 0x7fb0c0068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0d759e700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7fb0c0066420 0x7fb0c0068830 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.241 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.231+0000 7fb0d659c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb0c0060580 0x7fb0c0062a30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80644a0 tx=0x7fb0c8017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.242 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.235+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== osd.7 v2:172.21.15.154:6812/33722 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb0c0068ee0 con 0x7fb0c0066420 2023-10-07T23:30:37.261 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 --> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb0d80cab50 con 0x7fb0c0066420 2023-10-07T23:30:37.262 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0ceffd700 1 -- 172.21.15.57:0/2430633578 <== osd.7 v2:172.21.15.154:6812/33722 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb0d80cab50 con 0x7fb0c0066420 2023-10-07T23:30:37.262 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2023-10-07T23:30:37.262 INFO:teuthology.orchestra.run.smithi057.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-07T23:30:37.262 INFO:teuthology.orchestra.run.smithi057.stdout:} 2023-10-07T23:30:37.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7fb0c0066420 msgr2=0x7fb0c0068830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.263 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6812/33722,v1:172.21.15.154:6815/33722] conn(0x7fb0c0066420 0x7fb0c0068830 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb0c0060580 msgr2=0x7fb0c0062a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fb0c0060580 0x7fb0c0062a30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80644a0 tx=0x7fb0c8017000 comp rx=0 tx=0).stop 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 msgr2=0x7fb0d80c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 --2- 172.21.15.57:0/2430633578 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0d806cd40 0x7fb0d80c9c90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d890 tx=0x7fb0c4008040 comp rx=0 tx=0).stop 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0d759e700 1 -- 172.21.15.57:0/2430633578 reap_dead start 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 shutdown_connections 2023-10-07T23:30:37.264 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 >> 172.21.15.57:0/2430633578 conn(0x7fb0d8062ba0 msgr2=0x7fb0d81562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 shutdown_connections 2023-10-07T23:30:37.265 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.255+0000 7fb0dd20c700 1 -- 172.21.15.57:0/2430633578 wait complete. 2023-10-07T23:30:37.275 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-07T23:30:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 Processor -- start 2023-10-07T23:30:37.391 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 -- start start 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbef0158e60 0x7fbef006abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 0x7fbef00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef00caba0 con 0x7fbef00b04a0 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef006bb20 con 0x7fbef0158e60 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef6dde700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef006b560 con 0x7fbef0159230 2023-10-07T23:30:37.392 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef4b7a700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fbef0158e60 0x7fbef006abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef537b700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7fbef00b04a0 0x7fbef006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:50124/0 (socket says 172.21.15.57:50124) 2023-10-07T23:30:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbef537b700 1 -- 172.21.15.57:0/939820465 learned_addr learned my addr 172.21.15.57:0/939820465 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:37.393 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.383+0000 7fbeeffff700 1 --2- 172.21.15.57:0/939820465 >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 0x7fbef00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.394 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef4b7a700 1 -- 172.21.15.57:0/939820465 >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 msgr2=0x7fbef00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbeef7fe700 1 -- 172.21.15.57:0/939820465 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288849731 0 0) 0x7fbef00caba0 con 0x7fbef00b04a0 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef4b7a700 1 --2- 172.21.15.57:0/939820465 >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 0x7fbef00aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef4b7a700 1 -- 172.21.15.57:0/939820465 >> v1:172.21.15.57:6789/0 conn(0x7fbef00b04a0 legacy=0x7fbef006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef4b7a700 1 -- 172.21.15.57:0/939820465 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef006b840 con 0x7fbef0158e60 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbeeffff700 1 --2- 172.21.15.57:0/939820465 >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 0x7fbef00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef4b7a700 1 --2- 172.21.15.57:0/939820465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0158e60 0x7fbef006abc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbef006c010 tx=0x7fbee00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d6a21b9138327 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.395 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbeef7fe700 1 -- 172.21.15.57:0/939820465 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0011630 con 0x7fbef0158e60 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbeef7fe700 1 -- 172.21.15.57:0/939820465 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee0011790 con 0x7fbef0158e60 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbeef7fe700 1 -- 172.21.15.57:0/939820465 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee0011b30 con 0x7fbef0158e60 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- 172.21.15.57:0/939820465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0158e60 msgr2=0x7fbef006abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 --2- 172.21.15.57:0/939820465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0158e60 0x7fbef006abc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbef006c010 tx=0x7fbee00091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- 172.21.15.57:0/939820465 shutdown_connections 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 --2- 172.21.15.57:0/939820465 >> v2:172.21.15.154:3300/0 conn(0x7fbef0159230 0x7fbef00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 --2- 172.21.15.57:0/939820465 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0158e60 0x7fbef006abc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.396 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- 172.21.15.57:0/939820465 >> 172.21.15.57:0/939820465 conn(0x7fbef0068740 msgr2=0x7fbef0068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- 172.21.15.57:0/939820465 shutdown_connections 2023-10-07T23:30:37.397 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- 172.21.15.57:0/939820465 wait complete. 2023-10-07T23:30:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 Processor -- start 2023-10-07T23:30:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.387+0000 7fbef6dde700 1 -- start start 2023-10-07T23:30:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef00b04a0 0x7fbef00c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.399 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 0x7fbef0169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00c6ff0 con 0x7fbef0159230 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00c4300 con 0x7fbef00b04a0 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00c45e0 con 0x7fbef0158e60 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47010/0 (socket says 172.21.15.57:47010) 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 -- 172.21.15.57:0/2161582173 learned_addr learned my addr 172.21.15.57:0/2161582173 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:37.400 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbeeffff700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 0x7fbef0169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef4b7a700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef00b04a0 0x7fbef00c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 -- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 msgr2=0x7fbef0169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 0x7fbef0169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 -- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef00b04a0 msgr2=0x7fbef00c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef00b04a0 0x7fbef00c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 -- 172.21.15.57:0/2161582173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8009a20 con 0x7fbef0159230 2023-10-07T23:30:37.401 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbeeffff700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 0x7fbef0169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef537b700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c7e0 tx=0x7fbee8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8005c00 con 0x7fbef0159230 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8005d60 con 0x7fbef0159230 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee0009420 con 0x7fbef0159230 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004a80 con 0x7fbef0159230 2023-10-07T23:30:37.402 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.391+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef00c4ba0 con 0x7fbef0159230 2023-10-07T23:30:37.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef00647b0 con 0x7fbef0159230 2023-10-07T23:30:37.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fbee8005030 con 0x7fbef0159230 2023-10-07T23:30:37.403 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbeecff9700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 0x7fbed0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fbee807b970 con 0x7fbef0159230 2023-10-07T23:30:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbef4b7a700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 0x7fbed0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.404 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.395+0000 7fbef4b7a700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 0x7fbed0062a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbee00093f0 tx=0x7fbee0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.408 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.403+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbee8049be0 con 0x7fbef0159230 2023-10-07T23:30:37.569 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.563+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef00ca830 con 0x7fbed0060550 2023-10-07T23:30:37.570 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.563+0000 7fbeecff9700 1 -- 172.21.15.57:0/2161582173 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbef00ca830 con 0x7fbed0060550 2023-10-07T23:30:37.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 msgr2=0x7fbed0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.572 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 0x7fbed0062a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbee00093f0 tx=0x7fbee0017000 comp rx=0 tx=0).stop 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 msgr2=0x7fbef0169dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c7e0 tx=0x7fbee8007d90 comp rx=0 tx=0).stop 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 shutdown_connections 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbef0159230 0x7fbef0169dc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fbef0158e60 0x7fbef0169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fbed0060550 0x7fbed0062a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.573 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 --2- 172.21.15.57:0/2161582173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbef00b04a0 0x7fbef00c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 >> 172.21.15.57:0/2161582173 conn(0x7fbef0068740 msgr2=0x7fbef00cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 shutdown_connections 2023-10-07T23:30:37.574 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.567+0000 7fbef6dde700 1 -- 172.21.15.57:0/2161582173 wait complete. 2023-10-07T23:30:37.574 INFO:teuthology.orchestra.run.smithi057.stderr:instructed osd(s) 7 to deep-scrub 2023-10-07T23:30:37.585 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:30:37.691 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.683+0000 7fa3b8ff0700 1 Processor -- start 2023-10-07T23:30:37.692 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.683+0000 7fa3b8ff0700 1 -- start start 2023-10-07T23:30:37.693 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 0x7fa3b40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3b40cba50 0x7fa3b40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b40cac40 con 0x7fa3b40c6c10 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b40cc3e0 con 0x7fa3b40cba50 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40c7780 con 0x7fa3b40bee60 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3b40cba50 0x7fa3b40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7fa3b40cba50 0x7fa3b40cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47014/0 (socket says 172.21.15.57:47014) 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 -- 172.21.15.57:0/3521932661 learned_addr learned my addr 172.21.15.57:0/3521932661 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:37.694 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b259c700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 0x7fa3b40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 -- 172.21.15.57:0/3521932661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 msgr2=0x7fa3b40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 0x7fa3b40cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 -- 172.21.15.57:0/3521932661 >> v1:172.21.15.154:6789/0 conn(0x7fa3b40c6c10 legacy=0x7fa3b40c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 -- 172.21.15.57:0/3521932661 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b40cb610 con 0x7fa3b40cba50 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b259c700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 0x7fa3b40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b1d9b700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40cba50 0x7fa3b40cbea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009f20 tx=0x7fa3a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39180f76d0df1b43 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b159a700 1 -- 172.21.15.57:0/3521932661 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400e070 con 0x7fa3b40cba50 2023-10-07T23:30:37.695 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b159a700 1 -- 172.21.15.57:0/3521932661 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a400ba40 con 0x7fa3b40cba50 2023-10-07T23:30:37.696 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b159a700 1 -- 172.21.15.57:0/3521932661 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40092e0 con 0x7fa3b40cba50 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3521932661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40cba50 msgr2=0x7fa3b40cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40cba50 0x7fa3b40cbea0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009f20 tx=0x7fa3a4009840 comp rx=0 tx=0).stop 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3521932661 shutdown_connections 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40cba50 0x7fa3b40cbea0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3521932661 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40bee60 0x7fa3b40cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3521932661 >> 172.21.15.57:0/3521932661 conn(0x7fa3b40670e0 msgr2=0x7fa3b40674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.697 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3521932661 shutdown_connections 2023-10-07T23:30:37.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.687+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3521932661 wait complete. 2023-10-07T23:30:37.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 Processor -- start 2023-10-07T23:30:37.698 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 -- start start 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40c6c10 0x7fa3b40b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3b40cba50 0x7fa3b4161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40b0bb0 con 0x7fa3b40bee60 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40c5800 con 0x7fa3b40cba50 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40afcc0 con 0x7fa3b40c6c10 2023-10-07T23:30:37.700 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47028/0 (socket says 172.21.15.57:47028) 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 -- 172.21.15.57:0/3859671876 learned_addr learned my addr 172.21.15.57:0/3859671876 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b1d9b700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40c6c10 0x7fa3b40b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b2d9d700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3b40cba50 0x7fa3b4161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 -- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40c6c10 msgr2=0x7fa3b40b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40c6c10 0x7fa3b40b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 -- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3b40cba50 msgr2=0x7fa3b4161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.701 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3b40cba50 0x7fa3b4161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 -- 172.21.15.57:0/3859671876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4009ae0 con 0x7fa3b40bee60 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b259c700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c830 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8011600 con 0x7fa3b40bee60 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.691+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b40affa0 con 0x7fa3b40bee60 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b40b0560 con 0x7fa3b40bee60 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a8011970 con 0x7fa3b40bee60 2023-10-07T23:30:37.702 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8011ae0 con 0x7fa3b40bee60 2023-10-07T23:30:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b40711c0 con 0x7fa3b40bee60 2023-10-07T23:30:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7fa3a801a450 con 0x7fa3b40bee60 2023-10-07T23:30:37.703 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3a27fc700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 0x7fa394062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:37.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7fa3a807c8e0 con 0x7fa3b40bee60 2023-10-07T23:30:37.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.695+0000 7fa3b1d9b700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 0x7fa394062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:37.704 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.699+0000 7fa3b1d9b700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 0x7fa394062a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009880 tx=0x7fa3a4017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:37.711 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.703+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa3a804d100 con 0x7fa3b40bee60 2023-10-07T23:30:37.870 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.863+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3b40b0cf0 con 0x7fa394060550 2023-10-07T23:30:37.875 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.867+0000 7fa3a27fc700 1 -- 172.21.15.57:0/3859671876 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48626 (secure 0 0 0) 0x7fa3a4022070 con 0x7fa394060550 2023-10-07T23:30:37.876 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:37.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 msgr2=0x7fa394062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 0x7fa394062a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009880 tx=0x7fa3a4017000 comp rx=0 tx=0).stop 2023-10-07T23:30:37.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 msgr2=0x7fa3b40c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:37.877 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800c830 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2023-10-07T23:30:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 shutdown_connections 2023-10-07T23:30:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7fa394060550 0x7fa394062a00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.878 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3b40cba50 0x7fa3b4161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fa3b40c6c10 0x7fa3b40b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 --2- 172.21.15.57:0/3859671876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa3b40bee60 0x7fa3b40c2b70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 >> 172.21.15.57:0/3859671876 conn(0x7fa3b40670e0 msgr2=0x7fa3b40c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 shutdown_connections 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:37.871+0000 7fa3b8ff0700 1 -- 172.21.15.57:0/3859671876 wait complete. 2023-10-07T23:30:37.879 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:30:37.891 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-10-07T23:30:37.260000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2437608,"kb_used_data":6056,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752537112,"statfs":{"total":773094113280,"available":770598002688,"internally_reserved":0,"allocated":6201344,"data_stored":3206928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-151130,"num_objects":-34670,"num_object_clones":0,"num_object_copies":-69340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-34670,"num_whiteouts":0,"num_read":-342,"num_read_kb":0,"num_write":-38054,"num_write_kb":-34675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-168,"num_bytes_recovered":-645,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.657822"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-07T23:30:28.343243+0000","last_change":"2023-10-07T23:21:14.989362+0000","last_active":"2023-10-07T23:30:28.343243+0000","last_peered":"2023-10-07T23:30:28.343243+0000","last_clean":"2023-10-07T23:30:28.343243+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:30:28.343243+0000","last_undegraded":"2023-10-07T23:30:28.343243+0000","last_fullsized":"2023-10-07T23:30:28.343243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-07T23:21:13.814702+0000","last_clean_scrub_stamp":"2023-10-07T23:21:13.814702+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:56:38.784651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310768,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061072,"statfs":{"total":96636764160,"available":96318537728,"internally_reserved":0,"allocated":1220608,"data_stored":843642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":0.89400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0920000000000001,"5min":2.48,"15min":2.48},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"min":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.53900000000000003},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3240000000000001,"5min":1.409,"15min":1.4510000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.41899999999999998},"max":{"1min":1.286,"5min":1.397,"15min":1.5069999999999999},"last":0.64400000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.98699999999999999},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0269999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3460000000000001,"5min":1.494,"15min":1.7310000000000001},"last":1.0309999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.02,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.628,"15min":0.51100000000000001},"max":{"1min":1.7310000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0169999999999999,"15min":1.0109999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":2.7839999999999998},"last":0.93700000000000006}]},{"osd":4,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4390000000000001,"5min":1.569,"15min":1.603},"last":1.151},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0580000000000001,"15min":1.0860000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.554,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.2170000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.79},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1519999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.905},"last":1.099}]},{"osd":6,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7549999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.645,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.169}]}]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301996,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069844,"statfs":{"total":96636764160,"available":96327520256,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.80500000000000005,"15min":0.88200000000000001},"min":{"1min":0.50600000000000001,"5min":0.47199999999999998,"15min":0.435},"max":{"1min":1.3080000000000001,"5min":1.357,"15min":1.764},"last":1.1240000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90100000000000002,"15min":0.94999999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.6379999999999999},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78600000000000003,"15min":0.871},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.2190000000000001,"15min":1.5389999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.874,"15min":0.90700000000000003},"min":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.47299999999999998},"max":{"1min":1.2470000000000001,"5min":1.3129999999999999,"15min":1.726},"last":0.89600000000000002}]},{"osd":2,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.91900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.46999999999999997},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.878,"15min":0.93000000000000005},"min":{"1min":0.65300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.359,"5min":1.359,"15min":1.484},"last":0.97299999999999998}]},{"osd":3,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0289999999999999},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.48799999999999999},"max":{"1min":1.401,"5min":1.4279999999999999,"15min":1.6970000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0700000000000001,"15min":1.1080000000000001},"min":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.60899999999999999},"max":{"1min":1.379,"5min":1.5409999999999999,"15min":2.145},"last":1.2709999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91500000000000004,"15min":0.95999999999999996},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.4239999999999999,"5min":1.492,"15min":1.7529999999999999},"last":1.24},{"interface":"front","average":{"1min":1.054,"5min":0.97099999999999997,"15min":1.014},"min":{"1min":0.70399999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":2.2749999999999999},"last":1.0289999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98399999999999999,"15min":1.036},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.617},"last":1.079},{"interface":"front","average":{"1min":1.171,"5min":1.079,"15min":1.129},"min":{"1min":0.65100000000000002,"5min":0.56699999999999995,"15min":0.55300000000000005},"max":{"1min":1.599,"5min":1.599,"15min":1.8280000000000001},"last":1.351}]},{"osd":7,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.79900000000000004,"5min":0.63900000000000001,"15min":0.53600000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.167},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.143},"min":{"1min":0.755,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.653,"5min":1.653,"15min":1.671},"last":1.202}]}]},{"osd":5,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1779999999999999,"5min":1.4490000000000001,"15min":1.919},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.91000000000000003,"15min":0.98399999999999999},"min":{"1min":0.621,"5min":0.53300000000000003,"15min":0.42899999999999999},"max":{"1min":1.556,"5min":1.556,"15min":2.4950000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.89300000000000002},"min":{"1min":0.55900000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2310000000000001,"5min":1.306,"15min":2.3450000000000002},"last":1.4119999999999999},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.878,"15min":0.873},"min":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":3.6219999999999999,"5min":3.6219999999999999,"15min":3.6219999999999999},"last":1.0249999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.877,"15min":0.90600000000000003},"min":{"1min":0.54700000000000004,"5min":0.54100000000000004,"15min":0.47899999999999998},"max":{"1min":1.357,"5min":1.357,"15min":2.1549999999999998},"last":1.042},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0509999999999999},"min":{"1min":0.72299999999999998,"5min":0.63200000000000001,"15min":0.58899999999999997},"max":{"1min":1.2250000000000001,"5min":1.373,"15min":2.452},"last":1.167}]},{"osd":3,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.96099999999999997,"15min":0.98699999999999999},"min":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.51700000000000002},"max":{"1min":1.661,"5min":1.661,"15min":2.649},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97399999999999998,"15min":1.0229999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.46500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":2.077},"last":1.349}]},{"osd":4,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.131},{"interface":"front","average":{"1min":1.139,"5min":0.995,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.613,"5min":1.613,"15min":2.395},"last":1.22}]},{"osd":6,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98999999999999999,"15min":1.0840000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.45800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":3.1840000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.081,"15min":1.093},"min":{"1min":0.78500000000000003,"5min":0.61599999999999999,"15min":0.54700000000000004},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":1.258}]},{"osd":7,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.111,"15min":1.159},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"max":{"1min":1.708,"5min":1.708,"15min":2.7229999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.131,"15min":1.216},"min":{"1min":0.76700000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":3.3029999999999999},"last":1.514}]}]},{"osd":4,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79100000000000004,"15min":0.84199999999999997},"min":{"1min":0.53400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.671},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1830000000000001}]},{"osd":1,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.82399999999999995},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.1040000000000001,"5min":1.2729999999999999,"15min":1.5649999999999999},"last":1.032},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.871,"15min":0.88600000000000001},"min":{"1min":0.498,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2430000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.873,"15min":0.93200000000000005},"min":{"1min":0.57399999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.157,"5min":1.4430000000000001,"15min":1.952},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.073,"15min":1.137},"min":{"1min":0.71199999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.504,"5min":1.8620000000000001,"15min":2.0750000000000002},"last":0.99199999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.91600000000000004,"15min":0.94599999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.47799999999999998},"max":{"1min":1.288,"5min":1.6299999999999999,"15min":2.3330000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0049999999999999,"15min":1.038},"min":{"1min":0.72499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.496,"5min":1.496,"15min":1.7749999999999999},"last":1.077}]},{"osd":5,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.90600000000000003,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.47699999999999998,"15min":0.41399999999999998},"max":{"1min":1.532,"5min":1.744,"15min":1.744},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.046,"5min":0.98499999999999999,"15min":1.0549999999999999},"min":{"1min":0.54100000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4179999999999999,"5min":1.659,"15min":1.9630000000000001},"last":0.90900000000000003}]},{"osd":6,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.91400000000000003,"15min":0.97399999999999998},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2909999999999999,"5min":1.5349999999999999,"15min":1.706},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.052,"15min":1.1100000000000001},"min":{"1min":0.83699999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4219999999999999,"5min":1.792,"15min":2.3900000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.105,"15min":1.145},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.39000000000000001},"max":{"1min":1.5580000000000001,"5min":1.782,"15min":1.9379999999999999},"last":1.135},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0660000000000001,"15min":1.1160000000000001},"min":{"1min":0.745,"5min":0.68500000000000005,"15min":0.60899999999999999},"max":{"1min":1.3779999999999999,"5min":1.597,"15min":1.919},"last":0.88}]}]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306092,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065748,"statfs":{"total":96636764160,"available":96323325952,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.82999999999999996},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.33200000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":2.8769999999999998},"last":0.60299999999999998}]},{"osd":1,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.35599999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.64500000000000002,"5min":0.57199999999999995,"15min":0.34899999999999998},"max":{"1min":1.4099999999999999,"5min":1.4450000000000001,"15min":2.5369999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.875,"15min":0.80800000000000005},"min":{"1min":0.56299999999999994,"5min":0.51700000000000002,"15min":0.375},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.69699999999999995,"5min":0.64600000000000002,"15min":0.502},"max":{"1min":1.5289999999999999,"5min":1.665,"15min":2.7389999999999999},"last":0.94399999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64200000000000002},"max":{"1min":1.696,"5min":1.696,"15min":2.044},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0369999999999999,"15min":1.018},"min":{"1min":0.76200000000000001,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.704,"15min":1.704},"last":0.89000000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1040000000000001,"15min":1.0700000000000001},"min":{"1min":0.77700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5840000000000001,"5min":1.585,"15min":1.649},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.292,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.71999999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.6709999999999998},"last":0.71999999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2430000000000001,"15min":1.27},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.69999999999999996},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":2.802},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2330000000000001,"15min":1.2789999999999999},"min":{"1min":0.83899999999999997,"5min":0.72599999999999998,"15min":0.64000000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.758},"last":0.999}]},{"osd":7,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.92700000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7689999999999999,"5min":1.7709999999999999,"15min":2.2090000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.272,"5min":1.2270000000000001,"15min":1.202},"min":{"1min":0.93100000000000005,"5min":0.88,"15min":0.65900000000000003},"max":{"1min":1.6699999999999999,"5min":1.827,"15min":2.601},"last":1.0269999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":2.1739999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.34899999999999998},"max":{"1min":1.1259999999999999,"5min":1.381,"15min":1.827},"last":0.66100000000000003}]},{"osd":1,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":2.2690000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.32500000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.9019999999999999},"last":0.871}]},{"osd":3,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89700000000000002,"15min":0.85499999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0860000000000001,"15min":0.97999999999999998},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.44700000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.2229999999999999},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.071,"15min":1.0600000000000001},"min":{"1min":0.75800000000000001,"5min":0.71299999999999997,"15min":0.56699999999999995},"max":{"1min":1.587,"5min":1.853,"15min":2.0699999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.0960000000000001},"min":{"1min":0.68700000000000006,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.087,"15min":1.1080000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.484,"5min":1.657,"15min":2.302},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2150000000000001,"15min":1.1699999999999999},"min":{"1min":0.75,"5min":0.68200000000000005,"15min":0.64000000000000001},"max":{"1min":1.534,"5min":1.829,"15min":2.1819999999999999},"last":0.97999999999999998}]},{"osd":6,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.56000000000000005},"max":{"1min":1.6619999999999999,"5min":1.9199999999999999,"15min":2.2389999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1799999999999999,"15min":1.2130000000000001},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":2.4460000000000002},"last":1.097}]},{"osd":7,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.276,"15min":1.2689999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6579999999999999,"5min":1.8660000000000001,"15min":2.282},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.2030000000000001,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"max":{"1min":1.427,"5min":1.6359999999999999,"15min":10.183},"last":1.026}]}]},{"osd":0,"up_from":15,"seq":64424509556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":843642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.41399999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.55700000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.86299999999999999,"15min":0.82199999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.7370000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86399999999999999,"15min":0.81100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3759999999999999,"5min":1.611,"15min":1.611},"last":0.75900000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.78600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36799999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.52600000000000002},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.038,"15min":0.97499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.048,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.63300000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":1.095},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.554,"5min":1.7689999999999999,"15min":2.3879999999999999},"last":1.042},{"interface":"front","average":{"1min":0.996,"5min":1.028,"15min":1.0549999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.47999999999999998},"max":{"1min":1.3340000000000001,"5min":1.8799999999999999,"15min":2.1920000000000002},"last":0.85299999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2190000000000001,"15min":1.179},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.278},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.21,"15min":1.206},"min":{"1min":0.76700000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":1.8380000000000001,"15min":2.2759999999999998},"last":0.96299999999999997}]},{"osd":6,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.143,"15min":1.167},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4299999999999999,"5min":2.0910000000000002,"15min":2.7189999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2789999999999999,"15min":1.2589999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.165,"5min":2.165,"15min":2.375},"last":0.72799999999999998}]},{"osd":7,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.329,"15min":1.296},"min":{"1min":0.74299999999999999,"5min":0.73999999999999999,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":2.5030000000000001,"15min":2.524},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.22,"15min":1.23},"min":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5620000000000001,"5min":2.1030000000000002,"15min":2.3330000000000002},"last":0.93899999999999995}]}]},{"osd":1,"up_from":14,"seq":60129542261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.2130000000000001,"15min":2.5129999999999999},"last":0.495},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.33900000000000002},"max":{"1min":1.093,"5min":1.1419999999999999,"15min":1.1890000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94199999999999995,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.452,"15min":1.6439999999999999},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.92600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.42299999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.5089999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.0189999999999999,"15min":0.95699999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.42899999999999999},"max":{"1min":1.3440000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":1.798,"15min":1.798},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.046},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.478,"5min":1.603,"15min":1.603},"last":0.999}]},{"osd":5,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"max":{"1min":1.371,"5min":1.548,"15min":1.548},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.1719999999999999,"15min":1.1659999999999999},"min":{"1min":0.84299999999999997,"5min":0.745,"15min":0.53900000000000003},"max":{"1min":1.397,"5min":1.6859999999999999,"15min":1.8129999999999999},"last":0.876}]},{"osd":6,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.202},"min":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.57299999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.895},"last":1.071},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.22,"15min":1.2130000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.67200000000000004},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.802},"last":0.89900000000000002}]},{"osd":7,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.248,"15min":1.248},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.64500000000000002},"max":{"1min":1.3540000000000001,"5min":1.7809999999999999,"15min":1.8939999999999999},"last":1.022},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.27,"15min":1.27},"min":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.625},"max":{"1min":1.6930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.145}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:30:37.894 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-07T23:21:13.814702+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=7, tm_hour=23, tm_min=21, tm_sec=13, tm_wday=5, tm_yday=280, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=7, tm_hour=23, tm_min=30, tm_sec=32, tm_wday=5, tm_yday=280, tm_isdst=0) 2023-10-07T23:30:37.895 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-07T23:30:50.568 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:30:50.571 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:30:57.895 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-07T23:30:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.003+0000 7f8fc1999700 1 Processor -- start 2023-10-07T23:30:58.012 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.003+0000 7f8fc1999700 1 -- start start 2023-10-07T23:30:58.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 0x7f8fbc156520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc06b2f0 con 0x7f8fbc152720 2023-10-07T23:30:58.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc159330 con 0x7f8fbc152b90 2023-10-07T23:30:58.013 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc1598f0 con 0x7f8fbc068580 2023-10-07T23:30:58.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbbfff700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f8fbc152720 0x7f8fbc0ca220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36166/0 (socket says 172.21.15.57:36166) 2023-10-07T23:30:58.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbbfff700 1 -- 172.21.15.57:0/3429007505 learned_addr learned my addr 172.21.15.57:0/3429007505 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:58.014 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc0997700 1 --2- 172.21.15.57:0/3429007505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 0x7f8fbc156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888264118 0 0) 0x7f8fbc06b2f0 con 0x7f8fbc152720 2023-10-07T23:30:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fac003570 con 0x7f8fbc152720 2023-10-07T23:30:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 <== mon.1 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589717259 0 0) 0x7f8fbc159330 con 0x7f8fbc152b90 2023-10-07T23:30:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fbc06b2f0 con 0x7f8fbc152b90 2023-10-07T23:30:58.015 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc0997700 1 -- 172.21.15.57:0/3429007505 >> v1:172.21.15.154:6789/0 conn(0x7f8fbc152b90 legacy=0x7f8fbc06b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc0997700 1 -- 172.21.15.57:0/3429007505 >> v1:172.21.15.57:6789/0 conn(0x7f8fbc152720 legacy=0x7f8fbc0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc0997700 1 -- 172.21.15.57:0/3429007505 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc159610 con 0x7f8fbc068580 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc0997700 1 --2- 172.21.15.57:0/3429007505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 0x7f8fbc156520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009910 tx=0x7f8fb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62fe455dbb009350 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 <== mon.2 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00108a0 con 0x7f8fbc068580 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 <== mon.2 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0010a00 con 0x7f8fbc068580 2023-10-07T23:30:58.016 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fbb7fe700 1 -- 172.21.15.57:0/3429007505 <== mon.2 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0010d80 con 0x7f8fbc068580 2023-10-07T23:30:58.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- 172.21.15.57:0/3429007505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 msgr2=0x7f8fbc156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 --2- 172.21.15.57:0/3429007505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 0x7f8fbc156520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009910 tx=0x7f8fb00091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.017 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- 172.21.15.57:0/3429007505 shutdown_connections 2023-10-07T23:30:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 --2- 172.21.15.57:0/3429007505 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc068580 0x7f8fbc156520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- 172.21.15.57:0/3429007505 >> 172.21.15.57:0/3429007505 conn(0x7f8fbc0670e0 msgr2=0x7f8fbc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.007+0000 7f8fc1999700 1 -- 172.21.15.57:0/3429007505 shutdown_connections 2023-10-07T23:30:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 -- 172.21.15.57:0/3429007505 wait complete. 2023-10-07T23:30:58.018 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 Processor -- start 2023-10-07T23:30:58.019 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 -- start start 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fbc068580 0x7f8fbc13f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc152720 0x7f8fbc14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc14f1f0 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc14fd80 con 0x7f8fbc068580 2023-10-07T23:30:58.020 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1999700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c5650 con 0x7f8fbc152720 2023-10-07T23:30:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.011+0000 7f8fc1198700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52758/0 (socket says 172.21.15.57:52758) 2023-10-07T23:30:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 -- 172.21.15.57:0/2246777926 learned_addr learned my addr 172.21.15.57:0/2246777926 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc0997700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fbc068580 0x7f8fbc13f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.021 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fbbfff700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc152720 0x7f8fbc14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 -- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc152720 msgr2=0x7f8fbc14c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc152720 0x7f8fbc14c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.022 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 -- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fbc068580 msgr2=0x7f8fbc13f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fbc068580 0x7f8fbc13f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 -- 172.21.15.57:0/2246777926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0009420 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1198700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400c820 tx=0x7f8fb4002680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4012630 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc150020 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4012790 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.023 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4012b50 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc1505e0 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.015+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc150800 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.024 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.019+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f8fb401b400 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.019+0000 7f8fb8ff9700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 0x7f8fa0062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.019+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f8fb407c730 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.025 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.019+0000 7f8fc0997700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 0x7f8fa0062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.026 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.019+0000 7f8fc0997700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 0x7f8fa0062a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009fd0 tx=0x7f8fb0018000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.031+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8fb404cf60 con 0x7f8fbc14cbe0 2023-10-07T23:30:58.192 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.187+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 --> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fbc0711c0 con 0x7f8fa0060550 2023-10-07T23:30:58.198 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.191+0000 7f8fb8ff9700 1 -- 172.21.15.57:0/2246777926 <== mgr.4100 v2:172.21.15.154:6832/33383 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48391 (secure 0 0 0) 0x7f8fbc0711c0 con 0x7f8fa0060550 2023-10-07T23:30:58.199 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 msgr2=0x7f8fa0062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 0x7f8fa0062a00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009fd0 tx=0x7f8fb0018000 comp rx=0 tx=0).stop 2023-10-07T23:30:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 msgr2=0x7f8fbc0c32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400c820 tx=0x7f8fb4002680 comp rx=0 tx=0).stop 2023-10-07T23:30:58.202 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 shutdown_connections 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8fbc14cbe0 0x7f8fbc0c32d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f8fbc152720 0x7f8fbc14c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f8fa0060550 0x7f8fa0062a00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 --2- 172.21.15.57:0/2246777926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fbc068580 0x7f8fbc13f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 >> 172.21.15.57:0/2246777926 conn(0x7f8fbc0670e0 msgr2=0x7f8fbc0c0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 shutdown_connections 2023-10-07T23:30:58.203 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.195+0000 7f8fc1999700 1 -- 172.21.15.57:0/2246777926 wait complete. 2023-10-07T23:30:58.204 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2023-10-07T23:30:58.215 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2023-10-07T23:30:57.265301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2437608,"kb_used_data":6056,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752537112,"statfs":{"total":773094113280,"available":770598002688,"internally_reserved":0,"allocated":6201344,"data_stored":3206928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"1.0","version":"53'119","reported_seq":201,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-07T23:30:39.140412+0000","last_change":"2023-10-07T23:30:39.140336+0000","last_active":"2023-10-07T23:30:39.140412+0000","last_peered":"2023-10-07T23:30:39.140412+0000","last_clean":"2023-10-07T23:30:39.140412+0000","last_became_active":"2023-10-07T23:21:14.986931+0000","last_became_peered":"2023-10-07T23:21:14.986931+0000","last_unstale":"2023-10-07T23:30:39.140412+0000","last_undegraded":"2023-10-07T23:30:39.140412+0000","last_fullsized":"2023-10-07T23:30:39.140412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"53'119","last_scrub_stamp":"2023-10-07T23:30:39.140078+0000","last_deep_scrub":"53'119","last_deep_scrub_stamp":"2023-10-07T23:30:39.140078+0000","last_clean_scrub_stamp":"2023-10-07T23:30:39.140078+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:39.288137+0000","scrub_duration":0.045570303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509560,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310768,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061072,"statfs":{"total":96636764160,"available":96318537728,"internally_reserved":0,"allocated":1220608,"data_stored":843642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":0.89400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0920000000000001,"5min":2.48,"15min":2.48},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.92900000000000005,"15min":0.97699999999999998},"min":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.53900000000000003},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.091}]},{"osd":1,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3240000000000001,"5min":1.409,"15min":1.4510000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.90100000000000002},"min":{"1min":0.622,"5min":0.54600000000000004,"15min":0.41899999999999998},"max":{"1min":1.286,"5min":1.397,"15min":1.5069999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.94099999999999995,"15min":0.98699999999999999},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1830000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0269999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3460000000000001,"5min":1.494,"15min":1.7310000000000001},"last":1.0369999999999999}]},{"osd":3,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.02,"15min":0.98599999999999999},"min":{"1min":0.65000000000000002,"5min":0.628,"15min":0.51100000000000001},"max":{"1min":1.7310000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0169999999999999,"15min":1.0109999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":2.7839999999999998},"last":0.93400000000000005}]},{"osd":4,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4390000000000001,"5min":1.569,"15min":1.603},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0580000000000001,"15min":1.0860000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.554,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.206}]},{"osd":5,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.79},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.097,"15min":1.1519999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.905},"last":1.2470000000000001}]},{"osd":6,"last update":"Sat Oct 7 23:30:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1519999999999999,"15min":1.1839999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7549999999999999,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.294},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.645,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.319}]}]},{"osd":6,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301996,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069844,"statfs":{"total":96636764160,"available":96327520256,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.80500000000000005,"15min":0.88200000000000001},"min":{"1min":0.50600000000000001,"5min":0.47199999999999998,"15min":0.435},"max":{"1min":1.3080000000000001,"5min":1.357,"15min":1.764},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.90100000000000002,"15min":0.94999999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.6379999999999999},"last":1.03}]},{"osd":1,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78600000000000003,"15min":0.871},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.165,"5min":1.2190000000000001,"15min":1.5389999999999999},"last":1.083},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.874,"15min":0.90700000000000003},"min":{"1min":0.60199999999999998,"5min":0.56100000000000005,"15min":0.47299999999999998},"max":{"1min":1.2470000000000001,"5min":1.3129999999999999,"15min":1.726},"last":0.96199999999999997}]},{"osd":2,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.91900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.46999999999999997},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.878,"15min":0.93000000000000005},"min":{"1min":0.65300000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.359,"5min":1.359,"15min":1.484},"last":0.874}]},{"osd":3,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0049999999999999,"15min":1.0289999999999999},"min":{"1min":0.68200000000000005,"5min":0.58899999999999997,"15min":0.48799999999999999},"max":{"1min":1.401,"5min":1.4279999999999999,"15min":1.6970000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0700000000000001,"15min":1.1080000000000001},"min":{"1min":0.70199999999999996,"5min":0.65800000000000003,"15min":0.60899999999999999},"max":{"1min":1.379,"5min":1.5409999999999999,"15min":2.145},"last":1.2250000000000001}]},{"osd":4,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.91500000000000004,"15min":0.95999999999999996},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.4239999999999999,"5min":1.492,"15min":1.7529999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.054,"5min":0.97099999999999997,"15min":1.014},"min":{"1min":0.70399999999999996,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":2.2749999999999999},"last":1.1379999999999999}]},{"osd":5,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98399999999999999,"15min":1.036},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.544,"5min":1.544,"15min":1.617},"last":0.999},{"interface":"front","average":{"1min":1.171,"5min":1.079,"15min":1.129},"min":{"1min":0.65100000000000002,"5min":0.56699999999999995,"15min":0.55300000000000005},"max":{"1min":1.599,"5min":1.599,"15min":1.8280000000000001},"last":1.198}]},{"osd":7,"last update":"Sat Oct 7 23:30:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0940000000000001,"15min":1.1359999999999999},"min":{"1min":0.79900000000000004,"5min":0.63900000000000001,"15min":0.53600000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":2.0640000000000001},"last":1.284},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.143},"min":{"1min":0.755,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.653,"5min":1.653,"15min":1.671},"last":1.173}]}]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1779999999999999,"5min":1.4490000000000001,"15min":1.919},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.91000000000000003,"15min":0.98399999999999999},"min":{"1min":0.621,"5min":0.53300000000000003,"15min":0.42899999999999999},"max":{"1min":1.556,"5min":1.556,"15min":2.4950000000000001},"last":0.87}]},{"osd":1,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.89300000000000002},"min":{"1min":0.55900000000000005,"5min":0.496,"15min":0.496},"max":{"1min":1.2310000000000001,"5min":1.306,"15min":2.3450000000000002},"last":1.071},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.878,"15min":0.873},"min":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":3.6219999999999999,"5min":3.6219999999999999,"15min":3.6219999999999999},"last":0.83299999999999996}]},{"osd":2,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.877,"15min":0.90600000000000003},"min":{"1min":0.54700000000000004,"5min":0.54100000000000004,"15min":0.47899999999999998},"max":{"1min":1.357,"5min":1.357,"15min":2.1549999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0509999999999999},"min":{"1min":0.72299999999999998,"5min":0.63200000000000001,"15min":0.58899999999999997},"max":{"1min":1.2250000000000001,"5min":1.373,"15min":2.452},"last":1.2310000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.96099999999999997,"15min":0.98699999999999999},"min":{"1min":0.60199999999999998,"5min":0.59299999999999997,"15min":0.51700000000000002},"max":{"1min":1.661,"5min":1.661,"15min":2.649},"last":0.748},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97399999999999998,"15min":1.0229999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.46500000000000002},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":2.077},"last":1.1579999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.139,"5min":0.995,"15min":1.0569999999999999},"min":{"1min":0.80000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.613,"5min":1.613,"15min":2.395},"last":1.113}]},{"osd":6,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.98999999999999999,"15min":1.0840000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.45800000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":3.1840000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.081,"15min":1.093},"min":{"1min":0.78500000000000003,"5min":0.61599999999999999,"15min":0.54700000000000004},"max":{"1min":3.7090000000000001,"5min":3.7090000000000001,"15min":3.7090000000000001},"last":1.0249999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:30:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.111,"15min":1.159},"min":{"1min":0.69199999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"max":{"1min":1.708,"5min":1.708,"15min":2.7229999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.131,"15min":1.216},"min":{"1min":0.76700000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":3.3029999999999999},"last":1.3}]}]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79100000000000004,"15min":0.84199999999999997},"min":{"1min":0.53400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.671},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.92400000000000004,"15min":0.96999999999999997},"min":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78300000000000003,"15min":0.82399999999999995},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.1040000000000001,"5min":1.2729999999999999,"15min":1.5649999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.871,"15min":0.88600000000000001},"min":{"1min":0.498,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2430000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.98999999999999999}]},{"osd":2,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.873,"15min":0.93200000000000005},"min":{"1min":0.57399999999999995,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.157,"5min":1.4430000000000001,"15min":1.952},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.073,"15min":1.137},"min":{"1min":0.71199999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.504,"5min":1.8620000000000001,"15min":2.0750000000000002},"last":1.411}]},{"osd":3,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.91600000000000004,"15min":0.94599999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.47799999999999998},"max":{"1min":1.288,"5min":1.6299999999999999,"15min":2.3330000000000002},"last":1.159},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0049999999999999,"15min":1.038},"min":{"1min":0.72499999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.496,"5min":1.496,"15min":1.7749999999999999},"last":1.254}]},{"osd":5,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.90600000000000003,"15min":0.94599999999999995},"min":{"1min":0.5,"5min":0.47699999999999998,"15min":0.41399999999999998},"max":{"1min":1.532,"5min":1.744,"15min":1.744},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.046,"5min":0.98499999999999999,"15min":1.0549999999999999},"min":{"1min":0.54100000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4179999999999999,"5min":1.659,"15min":1.9630000000000001},"last":1.079}]},{"osd":6,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.91400000000000003,"15min":0.97399999999999998},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2909999999999999,"5min":1.5349999999999999,"15min":1.706},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.052,"15min":1.1100000000000001},"min":{"1min":0.83699999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.4219999999999999,"5min":1.792,"15min":2.3900000000000001},"last":1.3149999999999999}]},{"osd":7,"last update":"Sat Oct 7 23:30:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.105,"15min":1.145},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.39000000000000001},"max":{"1min":1.5580000000000001,"5min":1.782,"15min":1.9379999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0660000000000001,"15min":1.1160000000000001},"min":{"1min":0.745,"5min":0.68500000000000005,"15min":0.60899999999999999},"max":{"1min":1.3779999999999999,"5min":1.597,"15min":1.919},"last":1.212}]}]},{"osd":3,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306092,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065748,"statfs":{"total":96636764160,"available":96323325952,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67800000000000005,"15min":0.66800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.82999999999999996},"min":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.33200000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":2.8769999999999998},"last":0.57999999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77100000000000002,"15min":0.72099999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.35599999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.753},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94199999999999995,"15min":0.92300000000000004},"min":{"1min":0.64500000000000002,"5min":0.57199999999999995,"15min":0.34899999999999998},"max":{"1min":1.4099999999999999,"5min":1.4450000000000001,"15min":2.5369999999999999},"last":1.024}]},{"osd":2,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.875,"15min":0.80800000000000005},"min":{"1min":0.56299999999999994,"5min":0.51700000000000002,"15min":0.375},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96999999999999997,"15min":0.97999999999999998},"min":{"1min":0.69699999999999995,"5min":0.64600000000000002,"15min":0.502},"max":{"1min":1.5289999999999999,"5min":1.665,"15min":2.7389999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64200000000000002},"max":{"1min":1.696,"5min":1.696,"15min":2.044},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0369999999999999,"15min":1.018},"min":{"1min":0.76200000000000001,"5min":0.68600000000000005,"15min":0.65500000000000003},"max":{"1min":1.5649999999999999,"5min":1.704,"15min":1.704},"last":0.98299999999999998}]},{"osd":5,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1040000000000001,"15min":1.0700000000000001},"min":{"1min":0.77700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5840000000000001,"5min":1.585,"15min":1.649},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2050000000000001,"15min":1.228},"min":{"1min":0.71999999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.6709999999999998},"last":1.131}]},{"osd":6,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2430000000000001,"15min":1.27},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.69999999999999996},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":2.802},"last":1.202},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2330000000000001,"15min":1.2789999999999999},"min":{"1min":0.83899999999999997,"5min":0.72599999999999998,"15min":0.64000000000000001},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":2.758},"last":1.165}]},{"osd":7,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.92700000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7689999999999999,"5min":1.7709999999999999,"15min":2.2090000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2270000000000001,"15min":1.202},"min":{"1min":0.93100000000000005,"5min":0.88,"15min":0.65900000000000003},"max":{"1min":1.6699999999999999,"5min":1.827,"15min":2.601},"last":1.0629999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297900,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073940,"statfs":{"total":96636764160,"available":96331714560,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":2.1739999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.77300000000000002,"15min":0.78100000000000003},"min":{"1min":0.502,"5min":0.48699999999999999,"15min":0.34899999999999998},"max":{"1min":1.1259999999999999,"5min":1.381,"15min":1.827},"last":0.70899999999999996}]},{"osd":1,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.40899999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":2.2690000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.32500000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.9019999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.89700000000000002,"15min":0.85499999999999998},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.44700000000000001},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.875},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.0860000000000001,"15min":0.97999999999999998},"min":{"1min":0.70699999999999996,"5min":0.63300000000000001,"15min":0.44700000000000001},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.2229999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.071,"15min":1.0600000000000001},"min":{"1min":0.75800000000000001,"5min":0.71299999999999997,"15min":0.56699999999999995},"max":{"1min":1.587,"5min":1.853,"15min":2.0699999999999998},"last":1.022},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.0960000000000001},"min":{"1min":0.68700000000000006,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.94099999999999995}]},{"osd":5,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.087,"15min":1.1080000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.484,"5min":1.657,"15min":2.302},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2150000000000001,"15min":1.1699999999999999},"min":{"1min":0.75,"5min":0.68200000000000005,"15min":0.64000000000000001},"max":{"1min":1.534,"5min":1.829,"15min":2.1819999999999999},"last":1.1299999999999999}]},{"osd":6,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.56000000000000005},"max":{"1min":1.6619999999999999,"5min":1.9199999999999999,"15min":2.2389999999999999},"last":0.999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1799999999999999,"15min":1.2130000000000001},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5760000000000001,"5min":1.7130000000000001,"15min":2.4460000000000002},"last":1.087}]},{"osd":7,"last update":"Sat Oct 7 23:30:24 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.276,"15min":1.2689999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.6579999999999999,"5min":1.8660000000000001,"15min":2.282},"last":1.155},{"interface":"front","average":{"1min":1.129,"5min":1.2030000000000001,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.75600000000000001},"max":{"1min":1.427,"5min":1.6359999999999999,"15min":10.183},"last":1.181}]}]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302576,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069264,"statfs":{"total":96636764160,"available":96326926336,"internally_reserved":0,"allocated":1220608,"data_stored":843642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70399999999999996,"15min":0.68000000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.41399999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.216,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.85499999999999998}]},{"osd":2,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.86299999999999999,"15min":0.82199999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.7370000000000001},"last":0.751},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86399999999999999,"15min":0.81100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.3759999999999999,"5min":1.611,"15min":1.611},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.78600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36799999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.038,"15min":0.97499999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.048,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.83299999999999996}]},{"osd":4,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":1.095},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.554,"5min":1.7689999999999999,"15min":2.3879999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.996,"5min":1.028,"15min":1.0549999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.47999999999999998},"max":{"1min":1.3340000000000001,"5min":1.8799999999999999,"15min":2.1920000000000002},"last":1.2110000000000001}]},{"osd":5,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2190000000000001,"15min":1.179},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.278},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.21,"15min":1.206},"min":{"1min":0.76700000000000002,"5min":0.625,"15min":0.625},"max":{"1min":1.7010000000000001,"5min":1.8380000000000001,"15min":2.2759999999999998},"last":1.151}]},{"osd":6,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.143,"15min":1.167},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4299999999999999,"5min":2.0910000000000002,"15min":2.7189999999999999},"last":1.409},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2789999999999999,"15min":1.2589999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.165,"5min":2.165,"15min":2.375},"last":1.1850000000000001}]},{"osd":7,"last update":"Sat Oct 7 23:30:32 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.329,"15min":1.296},"min":{"1min":0.74299999999999999,"5min":0.73999999999999999,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":2.5030000000000001,"15min":2.524},"last":1.302},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.22,"15min":1.23},"min":{"1min":0.67800000000000005,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.5620000000000001,"5min":2.1030000000000002,"15min":2.3330000000000002},"last":1.3440000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310188,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061652,"statfs":{"total":96636764160,"available":96319131648,"internally_reserved":0,"allocated":626688,"data_stored":253274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.2130000000000001,"15min":2.5129999999999999},"last":0.747},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.33900000000000002},"max":{"1min":1.093,"5min":1.1419999999999999,"15min":1.1890000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.877},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94199999999999995,"15min":0.92200000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.452,"15min":1.6439999999999999},"last":1.077}]},{"osd":3,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93300000000000005,"15min":0.92600000000000005},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.42299999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.5089999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.0189999999999999,"15min":0.95699999999999996},"min":{"1min":0.61599999999999999,"5min":0.49199999999999999,"15min":0.42899999999999999},"max":{"1min":1.3440000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.0369999999999999}]},{"osd":4,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.50800000000000001},"max":{"1min":1.4279999999999999,"5min":1.798,"15min":1.798},"last":1.113},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.046},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.478,"5min":1.603,"15min":1.603},"last":1.139}]},{"osd":5,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.69999999999999996},"max":{"1min":1.371,"5min":1.548,"15min":1.548},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1719999999999999,"15min":1.1659999999999999},"min":{"1min":0.84299999999999997,"5min":0.745,"15min":0.53900000000000003},"max":{"1min":1.397,"5min":1.6859999999999999,"15min":1.8129999999999999},"last":1.335}]},{"osd":6,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.202},"min":{"1min":0.83099999999999996,"5min":0.82199999999999995,"15min":0.57299999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.895},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.22,"15min":1.2130000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.67200000000000004},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.802},"last":1.413}]},{"osd":7,"last update":"Sat Oct 7 23:30:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.248,"15min":1.248},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.64500000000000002},"max":{"1min":1.3540000000000001,"5min":1.7809999999999999,"15min":1.8939999999999999},"last":1.272},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.27,"15min":1.27},"min":{"1min":0.94899999999999995,"5min":0.93100000000000005,"15min":0.625},"max":{"1min":1.6930000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-07T23:30:58.218 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-07T23:30:58.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.319+0000 7f6aa9188700 1 Processor -- start 2023-10-07T23:30:58.327 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.319+0000 7f6aa9188700 1 -- start start 2023-10-07T23:30:58.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6aa406cf10 0x7f6aa415d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.328 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa40ca530 con 0x7f6aa4159010 2023-10-07T23:30:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa406ae60 con 0x7f6aa4158c40 2023-10-07T23:30:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa406b140 con 0x7f6aa406cf10 2023-10-07T23:30:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa37fe700 1 --1- >> v1:172.21.15.57:6789/0 conn(0x7f6aa4159010 0x7f6aa406c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.57:6789/0 says I am v1:172.21.15.57:36174/0 (socket says 172.21.15.57:36174) 2023-10-07T23:30:58.329 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa37fe700 1 -- 172.21.15.57:0/3257254418 learned_addr learned my addr 172.21.15.57:0/3257254418 (peer_addr_for_me v1:172.21.15.57:0/0) 2023-10-07T23:30:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa8987700 1 --2- 172.21.15.57:0/3257254418 >> v2:172.21.15.154:3300/0 conn(0x7f6aa406cf10 0x7f6aa415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa2ffd700 1 -- 172.21.15.57:0/3257254418 <== mon.0 v1:172.21.15.57:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580768107 0 0) 0x7f6aa40ca530 con 0x7f6aa4159010 2023-10-07T23:30:58.330 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa2ffd700 1 -- 172.21.15.57:0/3257254418 --> v1:172.21.15.57:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a8c003570 con 0x7f6aa4159010 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa8987700 1 -- 172.21.15.57:0/3257254418 >> v1:172.21.15.154:6789/0 conn(0x7f6aa4158c40 legacy=0x7f6aa40afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa8987700 1 -- 172.21.15.57:0/3257254418 >> v1:172.21.15.57:6789/0 conn(0x7f6aa4159010 legacy=0x7f6aa406c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa8987700 1 -- 172.21.15.57:0/3257254418 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa406b420 con 0x7f6aa406cf10 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa8987700 1 --2- 172.21.15.57:0/3257254418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa406cf10 0x7f6aa415d580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009e60 tx=0x7f6a9c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a289412f380a987f server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa2ffd700 1 -- 172.21.15.57:0/3257254418 <== mon.1 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00e070 con 0x7f6aa406cf10 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa2ffd700 1 -- 172.21.15.57:0/3257254418 <== mon.1 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c00bb60 con 0x7f6aa406cf10 2023-10-07T23:30:58.331 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa2ffd700 1 -- 172.21.15.57:0/3257254418 <== mon.1 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c012430 con 0x7f6aa406cf10 2023-10-07T23:30:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- 172.21.15.57:0/3257254418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa406cf10 msgr2=0x7f6aa415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.332 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 --2- 172.21.15.57:0/3257254418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa406cf10 0x7f6aa415d580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009e60 tx=0x7f6a9c009780 comp rx=0 tx=0).stop 2023-10-07T23:30:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- 172.21.15.57:0/3257254418 shutdown_connections 2023-10-07T23:30:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 --2- 172.21.15.57:0/3257254418 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa406cf10 0x7f6aa415d580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- 172.21.15.57:0/3257254418 >> 172.21.15.57:0/3257254418 conn(0x7f6aa4062ba0 msgr2=0x7f6aa4062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- 172.21.15.57:0/3257254418 shutdown_connections 2023-10-07T23:30:58.333 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.323+0000 7f6aa9188700 1 -- 172.21.15.57:0/3257254418 wait complete. 2023-10-07T23:30:58.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 Processor -- start 2023-10-07T23:30:58.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 -- start start 2023-10-07T23:30:58.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.334 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa4158c40 0x7f6aa40c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa4159010 0x7f6aa415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4159e30 con 0x7f6aa406cf10 2023-10-07T23:30:58.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4159540 con 0x7f6aa4158c40 2023-10-07T23:30:58.335 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa9188700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c9a80 con 0x7f6aa4159010 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa8987700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa4159010 0x7f6aa415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa37fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa4158c40 0x7f6aa40c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52772/0 (socket says 172.21.15.57:52772) 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 -- 172.21.15.57:0/4164492339 learned_addr learned my addr 172.21.15.57:0/4164492339 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 -- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa4159010 msgr2=0x7f6aa415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.337 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa4159010 0x7f6aa415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 -- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa4158c40 msgr2=0x7f6aa40c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa4158c40 0x7f6aa40c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.327+0000 7f6aa3fff700 1 -- 172.21.15.57:0/4164492339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c009a20 con 0x7f6aa406cf10 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6aa3fff700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c590 tx=0x7f6a9800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011590 con 0x7f6aa406cf10 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a98011710 con 0x7f6aa406cf10 2023-10-07T23:30:58.338 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa41597e0 con 0x7f6aa406cf10 2023-10-07T23:30:58.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98011ad0 con 0x7f6aa406cf10 2023-10-07T23:30:58.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa4153350 con 0x7f6aa406cf10 2023-10-07T23:30:58.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa40711c0 con 0x7f6aa406cf10 2023-10-07T23:30:58.339 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.331+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f6a9801b3b0 con 0x7f6aa406cf10 2023-10-07T23:30:58.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.335+0000 7f6a8bfff700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 0x7f6a90062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.335+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f6a9807b900 con 0x7f6aa406cf10 2023-10-07T23:30:58.340 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.335+0000 7f6aa37fe700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 0x7f6a90062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.341 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.335+0000 7f6aa37fe700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 0x7f6a90062a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6aa416c7a0 tx=0x7f6a94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.344 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.339+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6a9804c1c0 con 0x7f6aa406cf10 2023-10-07T23:30:58.508 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.503+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6aa406b700 con 0x7f6aa406cf10 2023-10-07T23:30:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.507+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6a980011f0 con 0x7f6aa406cf10 2023-10-07T23:30:58.516 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.507+0000 7f6a8bfff700 1 -- 172.21.15.57:0/4164492339 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6a9804f7e0 con 0x7f6aa406cf10 2023-10-07T23:30:58.518 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 msgr2=0x7f6a90062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 0x7f6a90062a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6aa416c7a0 tx=0x7f6a94009040 comp rx=0 tx=0).stop 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 msgr2=0x7f6aa4152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c590 tx=0x7f6a9800c8a0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 shutdown_connections 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f6aa4159010 0x7f6aa415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f6a90060550 0x7f6a90062a00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6aa4158c40 0x7f6aa40c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 --2- 172.21.15.57:0/4164492339 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6aa406cf10 0x7f6aa4152e10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 >> 172.21.15.57:0/4164492339 conn(0x7f6aa4062ba0 msgr2=0x7f6aa40cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 shutdown_connections 2023-10-07T23:30:58.520 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.511+0000 7f6aa9188700 1 -- 172.21.15.57:0/4164492339 wait complete. 2023-10-07T23:30:58.532 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-07T23:30:58.532 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-07T23:30:58.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.635+0000 7f98febe6700 1 Processor -- start 2023-10-07T23:30:58.642 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.635+0000 7f98febe6700 1 -- start start 2023-10-07T23:30:58.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f814fd80 0x7f98f806c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.645 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f98f80c93c0 0x7f98f80c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- --> v1:172.21.15.57:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f81594f0 con 0x7f98f8150150 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- --> v2:172.21.15.57:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f80c9da0 con 0x7f98f80c93c0 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8159790 con 0x7f98f814fd80 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f98f80c93c0 0x7f98f80c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 --2- >> v2:172.21.15.57:3300/0 conn(0x7f98f80c93c0 0x7f98f80c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52782/0 (socket says 172.21.15.57:52782) 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 -- 172.21.15.57:0/409155976 learned_addr learned my addr 172.21.15.57:0/409155976 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fdbe4700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f814fd80 0x7f98f806c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.646 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 -- 172.21.15.57:0/409155976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f814fd80 msgr2=0x7f98f806c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-07T23:30:58.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f814fd80 0x7f98f806c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 -- 172.21.15.57:0/409155976 >> v1:172.21.15.57:6789/0 conn(0x7f98f8150150 legacy=0x7f98f80abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 -- 172.21.15.57:0/409155976 --> v2:172.21.15.57:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8159a70 con 0x7f98f80c93c0 2023-10-07T23:30:58.647 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fe3e5700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f80c93c0 0x7f98f80c9860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009f20 tx=0x7f98f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6475ce2c528fcc02 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fcbe2700 1 -- 172.21.15.57:0/409155976 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000e070 con 0x7f98f80c93c0 2023-10-07T23:30:58.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fcbe2700 1 -- 172.21.15.57:0/409155976 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f98f000b9d0 con 0x7f98f80c93c0 2023-10-07T23:30:58.648 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98fcbe2700 1 -- 172.21.15.57:0/409155976 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00092e0 con 0x7f98f80c93c0 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- 172.21.15.57:0/409155976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f80c93c0 msgr2=0x7f98f80c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f80c93c0 0x7f98f80c9860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009f20 tx=0x7f98f0009840 comp rx=0 tx=0).stop 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- 172.21.15.57:0/409155976 shutdown_connections 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f80c93c0 0x7f98f80c9860 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 --2- 172.21.15.57:0/409155976 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f814fd80 0x7f98f806c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- 172.21.15.57:0/409155976 >> 172.21.15.57:0/409155976 conn(0x7f98f8067b80 msgr2=0x7f98f80c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- 172.21.15.57:0/409155976 shutdown_connections 2023-10-07T23:30:58.649 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.639+0000 7f98febe6700 1 -- 172.21.15.57:0/409155976 wait complete. 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 Processor -- start 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 -- start start 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f80c93c0 0x7f98f814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 0x7f98f8068d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8150910 con 0x7f98f8150150 2023-10-07T23:30:58.652 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f816c760 con 0x7f98f814fd80 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f816c9b0 con 0x7f98f80c93c0 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52798/0 (socket says 172.21.15.57:52798) 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 -- 172.21.15.57:0/2685716268 learned_addr learned my addr 172.21.15.57:0/2685716268 (peer_addr_for_me v2:172.21.15.57:0/0) 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fdbe4700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f80c93c0 0x7f98f814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fd3e3700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 0x7f98f8068d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.653 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 -- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f80c93c0 msgr2=0x7f98f814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f80c93c0 0x7f98f814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 -- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 msgr2=0x7f98f8068d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 0x7f98f8068d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 -- 172.21.15.57:0/2685716268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009ae0 con 0x7f98f8150150 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fd3e3700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 0x7f98f8068d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-07T23:30:58.654 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98fe3e5700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98f0008740 tx=0x7f98f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0008be0 con 0x7f98f8150150 2023-10-07T23:30:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f98f0008d60 con 0x7f98f8150150 2023-10-07T23:30:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.643+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f816cc90 con 0x7f98f8150150 2023-10-07T23:30:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00256d0 con 0x7f98f8150150 2023-10-07T23:30:58.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f816d250 con 0x7f98f8150150 2023-10-07T23:30:58.656 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f816d470 con 0x7f98f8150150 2023-10-07T23:30:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) v1 ==== 79959+0+0 (secure 0 0 0) 0x7f98f0003040 con 0x7f98f8150150 2023-10-07T23:30:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98edffb700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 0x7f98dc062a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-07T23:30:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4982+0+0 (secure 0 0 0) 0x7f98f007e120 con 0x7f98f8150150 2023-10-07T23:30:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.647+0000 7f98fdbe4700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 0x7f98dc062a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-07T23:30:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.651+0000 7f98fdbe4700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 0x7f98dc062a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f98f80ab240 tx=0x7f98f40091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-07T23:30:58.662 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.655+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98f004ea60 con 0x7f98f8150150 2023-10-07T23:30:58.859 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.851+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98f80711c0 con 0x7f98f8150150 2023-10-07T23:30:58.860 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98edffb700 1 -- 172.21.15.57:0/2685716268 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f98f0020de0 con 0x7f98f8150150 2023-10-07T23:30:58.861 INFO:teuthology.orchestra.run.smithi057.stdout: 2023-10-07T23:30:58.861 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5048,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/2647269057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2647269057},{"type":"v1","addr":"172.21.15.154:6837","nonce":2647269057}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6833/2014638394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":2014638394},{"type":"v1","addr":"172.21.15.57:6833","nonce":2014638394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/3534360661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":3534360661},{"type":"v1","addr":"172.21.15.57:6835","nonce":3534360661}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":5683,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/3437554138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3437554138},{"type":"v1","addr":"172.21.15.154:6835","nonce":3437554138}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[]} 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 msgr2=0x7f98dc062a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 0x7f98dc062a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f98f80ab240 tx=0x7f98f40091a0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 msgr2=0x7f98f8069250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98f0008740 tx=0x7f98f0009840 comp rx=0 tx=0).stop 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 shutdown_connections 2023-10-07T23:30:58.863 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98f8150150 0x7f98f8069250 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:6832/33383,v1:172.21.15.154:6833/33383] conn(0x7f98dc060550 0x7f98dc062a00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98f814fd80 0x7f98f8068d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 --2- 172.21.15.57:0/2685716268 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f98f80c93c0 0x7f98f814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 >> 172.21.15.57:0/2685716268 conn(0x7f98f8067b80 msgr2=0x7f98f80ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 shutdown_connections 2023-10-07T23:30:58.864 INFO:teuthology.orchestra.run.smithi057.stderr:2023-10-07T23:30:58.855+0000 7f98febe6700 1 -- 172.21.15.57:0/2685716268 wait complete. 2023-10-07T23:30:58.865 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 22 2023-10-07T23:30:58.874 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-07T23:30:58.874 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-07T23:30:58.875 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-07T23:30:58.875 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:58.891 INFO:tasks.ceph.mds.a:Stopped 2023-10-07T23:30:58.891 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-07T23:30:58.891 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:58.904 INFO:tasks.ceph.mds.c:Stopped 2023-10-07T23:30:58.904 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-07T23:30:58.905 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:58.921 INFO:tasks.ceph.mds.b:Stopped 2023-10-07T23:30:58.921 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-07T23:30:58.921 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:58.936 INFO:tasks.ceph.mds.d:Stopped 2023-10-07T23:30:58.936 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-07T23:30:58.937 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-07T23:30:58.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.038 INFO:tasks.ceph.osd.0:Stopped 2023-10-07T23:30:59.039 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-07T23:30:59.039 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.115 INFO:tasks.ceph.osd.1:Stopped 2023-10-07T23:30:59.115 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-07T23:30:59.115 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.178 INFO:tasks.ceph.osd.2:Stopped 2023-10-07T23:30:59.178 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-07T23:30:59.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.259 INFO:tasks.ceph.osd.3:Stopped 2023-10-07T23:30:59.260 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-07T23:30:59.260 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.347 INFO:tasks.ceph.osd.4:Stopped 2023-10-07T23:30:59.349 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-07T23:30:59.350 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.443 INFO:tasks.ceph.osd.5:Stopped 2023-10-07T23:30:59.444 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-07T23:30:59.444 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.506 INFO:tasks.ceph.osd.6:Stopped 2023-10-07T23:30:59.507 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-07T23:30:59.507 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.589 INFO:tasks.ceph.osd.7:Stopped 2023-10-07T23:30:59.590 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-07T23:30:59.590 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-07T23:30:59.590 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.622 INFO:tasks.ceph.mgr.y:Stopped 2023-10-07T23:30:59.623 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-07T23:30:59.623 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.650 INFO:tasks.ceph.mgr.x:Stopped 2023-10-07T23:30:59.650 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-07T23:30:59.651 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-07T23:30:59.651 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.671 INFO:tasks.ceph.mon.a:Stopped 2023-10-07T23:30:59.672 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-07T23:30:59.672 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.690 INFO:tasks.ceph.mon.b:Stopped 2023-10-07T23:30:59.691 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-07T23:30:59.691 INFO:teuthology.orchestra.run:waiting for 300 2023-10-07T23:30:59.751 INFO:tasks.ceph.mon.c:Stopped 2023-10-07T23:30:59.752 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-07T23:30:59.752 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2023-10-07T23:30:59.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:30:59.779 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-07T23:31:00.121 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:31:00.122 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-07T23:31:00.408 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:31:00.409 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-07T23:31:00.607 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:31:00.608 DEBUG:teuthology.orchestra.run.smithi057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-07T23:31:00.767 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:31:00.767 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-07T23:31:01.451 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:31:01.452 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-07T23:31:01.664 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:31:01.665 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-07T23:31:01.836 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:31:01.837 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-07T23:31:02.023 INFO:tasks.ceph:Archiving mon data... 2023-10-07T23:31:02.024 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/data/mon.a.tgz 2023-10-07T23:31:02.024 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2023-10-07T23:31:02.031 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.oTxfy6xEON 2023-10-07T23:31:02.031 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oTxfy6xEON 2023-10-07T23:31:02.104 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.oTxfy6xEON 2023-10-07T23:31:02.129 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.oTxfy6xEON is 271KB 2023-10-07T23:31:02.194 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.oTxfy6xEON 2023-10-07T23:31:02.201 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/data/mon.b.tgz 2023-10-07T23:31:02.201 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-07T23:31:02.208 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.g2lKWZHBaC 2023-10-07T23:31:02.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.g2lKWZHBaC 2023-10-07T23:31:02.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.g2lKWZHBaC 2023-10-07T23:31:02.353 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.g2lKWZHBaC is 280KB 2023-10-07T23:31:02.412 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.g2lKWZHBaC 2023-10-07T23:31:02.419 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/data/mon.c.tgz 2023-10-07T23:31:02.419 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-10-07T23:31:02.425 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.gZUlRV38h2 2023-10-07T23:31:02.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gZUlRV38h2 2023-10-07T23:31:02.500 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.gZUlRV38h2 2023-10-07T23:31:02.518 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.gZUlRV38h2 is 279KB 2023-10-07T23:31:02.580 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.gZUlRV38h2 2023-10-07T23:31:02.587 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-07T23:31:02.587 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-07T23:31:02.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-07T23:31:02.605 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-07T23:31:02.605 DEBUG:teuthology.orchestra.run.smithi057:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-07T23:31:02.649 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-07T23:31:02.678 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-07T23:31:02.687 INFO:teuthology.orchestra.run.smithi057.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-07T23:31:02.694 INFO:tasks.ceph:Archiving crash dumps... 2023-10-07T23:31:02.696 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi057/crash 2023-10-07T23:31:02.697 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-07T23:31:02.713 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi154/crash 2023-10-07T23:31:02.714 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-07T23:31:02.730 INFO:tasks.ceph:Shutting down logrotate 2023-10-07T23:31:02.731 DEBUG:teuthology.orchestra.run.smithi057:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:31:02.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-07T23:31:02.816 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-07T23:31:02.831 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-07T23:31:02.847 INFO:tasks.ceph:Compressing logs... 2023-10-07T23:31:02.847 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T23:31:02.849 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-07T23:31:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-07T23:31:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-07T23:31:02.870 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39634.log 2023-10-07T23:31:02.870 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39796.log 2023-10-07T23:31:02.871 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39356.log 2023-10-07T23:31:02.872 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.39634.log: /var/log/ceph/ceph-client.1.39796.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-07T23:31:02.873 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.1.39356.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44414.log 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42160.log 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38787.log 2023-10-07T23:31:02.875 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2023-10-07T23:31:02.876 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42160.log: /var/log/ceph/ceph-client.admin.38787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2023-10-07T23:31:02.876 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44414.log.gz 2023-10-07T23:31:02.876 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38787.log.gz 2023-10-07T23:31:02.876 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2023-10-07T23:31:02.878 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.5.log: 92.9% -- replaced with /var/log/ceph/ceph-client.1.39634.log.gz 2023-10-07T23:31:02.879 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-07T23:31:02.879 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.1.39356.log.gz 2023-10-07T23:31:02.879 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-07T23:31:02.879 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33573.log 2023-10-07T23:31:02.879 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-07T23:31:02.882 INFO:teuthology.orchestra.run.smithi057.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2023-10-07T23:31:02.882 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41290.log 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40731.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40920.log 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44627.log 2023-10-07T23:31:02.883 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43829.log 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40920.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41290.log.gz 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44627.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40920.log.gz 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.44627.log.gz 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.43271.log 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42033.log 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38463.log 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43271.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43829.log.gz 2023-10-07T23:31:02.884 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42528.log 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43271.log.gz 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41736.log 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42033.log.gz 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38463.log: /var/log/ceph/ceph-client.admin.42528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38463.log.gz 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42528.log.gz 2023-10-07T23:31:02.885 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2023-10-07T23:31:02.887 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33573.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33573.log.gz 2023-10-07T23:31:02.887 INFO:teuthology.orchestra.run.smithi154.stderr: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-07T23:31:02.887 INFO:teuthology.orchestra.run.smithi154.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.1.39796.log.gz 2023-10-07T23:31:02.887 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33545.log 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39469.log 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33545.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33545.log.gz 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.3.39469.log: 88.5% -- replaced with /var/log/ceph/ceph-client.3.39469.log.gz 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-07T23:31:02.888 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33600.log 2023-10-07T23:31:02.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-07T23:31:02.897 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-07T23:31:02.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33600.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33600.log.gz 2023-10-07T23:31:02.902 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33628.log 2023-10-07T23:31:02.905 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42466.log 2023-10-07T23:31:02.905 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44897.log: /var/log/ceph/ceph-client.admin.41572.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41736.log.gz 2023-10-07T23:31:02.905 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45140.log: /var/log/ceph/ceph-client.admin.38242.log: gzip -5 --verbose -- 89.4%/var/log/ceph/ceph-client.admin.42096.log 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42466.log.gz 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2023-10-07T23:31:02.906 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43082.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42096.log.gz 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44252.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 /var/log/ceph/ceph-client.admin.39813.log: --verbose -- /var/log/ceph/ceph-client.admin.44735.log 2023-10-07T23:31:02.907 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr: 90.8%/var/log/ceph/ceph-client.admin.44735.log: -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.43620.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44735.log.gz 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38571.log 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43190.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39208.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-07T23:31:02.908 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40528.log 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38571.log.gz 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38378.log 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44306.log 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40528.log: /var/log/ceph/ceph-client.admin.38378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38215.log 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40528.log.gz 2023-10-07T23:31:02.909 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38378.log.gz 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38517.log 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44306.log.gz 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40366.log 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38517.log: /var/log/ceph/ceph-client.admin.40366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42385.log 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38215.log.gz 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38517.log.gz 2023-10-07T23:31:02.910 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43647.log 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42674.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40366.log.gz 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38297.log 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42385.log.gz 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39786.log 2023-10-07T23:31:02.911 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42439.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43647.log.gz 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38297.log.gz 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44654.log 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39786.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8%/var/log/ceph/ceph-client.admin.39218.log: -- replaced with /var/log/ceph/ceph-client.admin.39786.log.gz 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38998.log 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2023-10-07T23:31:02.912 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40785.log 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44654.log: /var/log/ceph/ceph-client.admin.38998.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44654.log.gz 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42250.log 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38998.log.gz 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40785.log.gz 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39705.log 2023-10-07T23:31:02.913 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42250.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38598.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42250.log.gz 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42968.log 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.39205.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44171.log 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39705.log.gz 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2023-10-07T23:31:02.914 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40636.log 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44171.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42968.log.gz 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39220.log 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40636.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44171.log.gz 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40636.log.gz 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-07T23:31:02.915 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39220.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39220.log.gz 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39001.log 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39624.log 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44681.log 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43566.log 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39624.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39001.log.gz 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39624.log.gz 2023-10-07T23:31:02.916 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38783.log 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28423.log 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44681.log.gz 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.43566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38783.log: /var/log/ceph/ceph.tmp-client.admin.28423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41344.log 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38783.log.gz 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28423.log.gz 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45032.log 2023-10-07T23:31:02.917 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.0.43566.log.gz 2023-10-07T23:31:02.918 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7%/var/log/ceph/ceph-client.admin.41344.log: -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2023-10-07T23:31:02.918 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41344.log.gz 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.4.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33520.log 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33628.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33628.log.gz 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33656.log 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33520.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33520.log.gz 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33656.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33656.log.gz 2023-10-07T23:31:02.923 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-07T23:31:02.924 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.33495.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2023-10-07T23:31:02.924 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33681.log 2023-10-07T23:31:02.933 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.33681.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33681.log.gz 2023-10-07T23:31:02.939 INFO:teuthology.orchestra.run.smithi057.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-07T23:31:02.939 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38999.log 2023-10-07T23:31:02.939 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45032.log: /var/log/ceph/ceph-client.admin.38999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45032.log.gz 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38999.log.gz 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40947.log 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40196.log 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44279.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2023-10-07T23:31:02.940 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44441.log 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40947.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44762.log 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40196.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40947.log.gz 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41599.log 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40196.log.gz 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44441.log.gz 2023-10-07T23:31:02.941 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44762.log.gz 2023-10-07T23:31:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39732.log 2023-10-07T23:31:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38784.log 2023-10-07T23:31:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38490.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41599.log.gz 2023-10-07T23:31:02.942 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42006.log 2023-10-07T23:31:02.942 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39732.log.gz 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42941.log 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42647.log 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38784.log.gz 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42006.log.gz 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.42941.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43244.log 2023-10-07T23:31:02.943 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42647.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42941.log.gz 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39219.log 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42647.log.gz 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42887.log 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43244.log.gz 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39219.log: gzip -5 /var/log/ceph/ceph-client.admin.42887.log: --verbose -- /var/log/ceph/ceph-client.admin.39678.log 2023-10-07T23:31:02.944 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39219.log.gz 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42787.log 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42887.log.gz 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.39678.log: -- /var/log/ceph/ceph-client.0.41926.log 88.3% 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45223.log 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39678.log.gz 2023-10-07T23:31:02.945 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.41926.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42787.log.gz 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41682.log 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45223.log.gz 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.43460.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38113.log 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41682.log: /var/log/ceph/ceph-client.admin.41263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44924.log 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41682.log.gz 2023-10-07T23:31:02.946 INFO:teuthology.orchestra.run.smithi057.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38147.log 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38113.log.gz 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39408.log 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38147.log: /var/log/ceph/ceph-client.admin.44144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41763.log 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44924.log.gz 2023-10-07T23:31:02.947 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38147.log.gz 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40893.log 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39408.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38788.log 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39408.log.gz 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41763.log.gz 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42331.log 2023-10-07T23:31:02.948 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38788.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40893.log.gz 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42277.log 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38788.log.gz 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42331.log: /var/log/ceph/ceph-client.admin.42277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42331.log.gz 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1%gzip /var/log/ceph/ceph-client.admin.45059.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42566.log 2023-10-07T23:31:02.949 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42277.log.gz 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.38324.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42566.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.40501.log: -- /var/log/ceph/ceph-client.admin.33392.log 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42566.log.gz 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38995.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2023-10-07T23:31:02.950 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.33392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33392.log.gz 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43109.log 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log /var/log/ceph/ceph-client.admin.44522.log: 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45113.log 2023-10-07T23:31:02.951 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43109.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43109.log.gz 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44468.log: /var/log/ceph/ceph-client.admin.45113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0%/var/log/ceph/ceph-client.admin.39202.log: -- replaced with /var/log/ceph/ceph-client.admin.45113.log.gz 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2023-10-07T23:31:02.952 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38779.log 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41481.log 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43379.log: /var/log/ceph/ceph-client.admin.38779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39002.log 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr: 89.2%/var/log/ceph/ceph-client.admin.41481.log: -- replaced with /var/log/ceph/ceph-client.admin.38779.log.gz 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43325.log 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41481.log.gz 2023-10-07T23:31:02.953 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40447.log 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43406.log 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39002.log.gz 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43325.log: /var/log/ceph/ceph-client.admin.40447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38056.log 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43325.log.gz 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.43406.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45171.log 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40447.log.gz 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2023-10-07T23:31:02.954 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43406.log.gz 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38056.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45171.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38056.log.gz 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41868.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45171.log.gz 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.41115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44360.log 2023-10-07T23:31:02.955 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39597.log /var/log/ceph/ceph-client.admin.43679.log: 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44360.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41317.log 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44360.log.gz 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41182.log 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 /var/log/ceph/ceph-client.admin.41317.log: --verbose -- /var/log/ceph/ceph-client.admin.40690.log 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39597.log.gz 2023-10-07T23:31:02.956 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41182.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41317.log.gz 2023-10-07T23:31:02.957 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41182.log.gz 2023-10-07T23:31:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42412.log 2023-10-07T23:31:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.40690.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44063.log 2023-10-07T23:31:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44225.log 2023-10-07T23:31:02.968 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42412.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40690.log.gz 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44063.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42412.log.gz 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44063.log.gz 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38782.log 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39435.log 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39489.log: -- replaced with /var/log/ceph/ceph-client.admin.44225.log.gz 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38782.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2023-10-07T23:31:02.969 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41028.log 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4%gzip /var/log/ceph/ceph-client.admin.39435.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43856.log 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38782.log.gz 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39435.log.gz 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42128.log 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43738.log 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43856.log: -- replaced with /var/log/ceph/ceph-client.admin.41028.log.gz 2023-10-07T23:31:02.970 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42128.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43856.log.gz 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39462.log 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42128.log.gz 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.43738.log: /var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43997.log 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz/var/log/ceph/ceph-client.admin.39462.log: 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39462.log.gz 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43997.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43997.log.gz 2023-10-07T23:31:02.971 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43136.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44333.log 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43511.log 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44333.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44333.log.gz 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41055.log 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40582.log 2023-10-07T23:31:02.972 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43511.log: /var/log/ceph/ceph-client.admin.41055.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43511.log.gz 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38679.log 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41055.log.gz 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40582.log.gz 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38679.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38679.log.gz 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42593.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2023-10-07T23:31:02.973 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40028.log 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39978.log 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40866.log: /var/log/ceph/ceph-client.admin.40028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42493.log 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40028.log.gz 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.0.39978.log: 90.6% -- replaced with /var/log/ceph/ceph-client.0.39978.log.gz 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41790.log 2023-10-07T23:31:02.974 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42493.log: /var/log/ceph/ceph-client.admin.41790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42493.log.gz 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41790.log.gz 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43163.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40474.log 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41452.log: /var/log/ceph/ceph-client.admin.40474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2023-10-07T23:31:02.975 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40229.log 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40474.log.gz 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39003.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40229.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40229.log.gz 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44198.log 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38706.log 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44870.log: /var/log/ceph/ceph-client.admin.44198.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz 2023-10-07T23:31:02.976 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39759.log 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38706.log: /var/log/ceph/ceph-client.admin.44708.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38706.log.gz 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38270.log 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39759.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39759.log.gz 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38652.log 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38270.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38270.log.gz 2023-10-07T23:31:02.977 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41209.log 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38652.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38652.log.gz 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41209.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41209.log.gz 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42995.log 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43298.log: /var/log/ceph/ceph-client.admin.44387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41398.log 2023-10-07T23:31:02.978 INFO:teuthology.orchestra.run.smithi057.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42995.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42995.log.gz 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40420.log 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.41398.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41977.log 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40420.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41398.log.gz 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40974.log 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40420.log.gz 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41977.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41977.log.gz 2023-10-07T23:31:02.979 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2023-10-07T23:31:02.980 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-07T23:31:02.980 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40974.log: /var/log/ceph/ceph-client.admin.39651.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40974.log.gz 2023-10-07T23:31:02.980 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42223.log 2023-10-07T23:31:02.980 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph.log: 84.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40609.log 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.42223.log: -- /var/log/ceph/ceph-client.admin.40663.log 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40609.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42223.log.gz 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42304.log 2023-10-07T23:31:02.986 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40609.log.gz 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40663.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40663.log.gz 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42304.log: /var/log/ceph/ceph-client.admin.42914.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42304.log.gz 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44090.log 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44843.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2023-10-07T23:31:02.987 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41082.log 2023-10-07T23:31:02.989 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44090.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44090.log.gz 2023-10-07T23:31:02.989 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-07T23:31:02.989 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41510.log 2023-10-07T23:31:02.989 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41082.log: /var/log/ceph/ceph-osd.0.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41082.log.gz 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41510.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41510.log.gz 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43883.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41628.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39004.log 2023-10-07T23:31:02.990 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40339.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45086.log 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39004.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39004.log.gz 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5%/var/log/ceph/ceph-client.admin.44951.log: -- replaced with /var/log/ceph/ceph-client.admin.45086.log.gz 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41371.log 2023-10-07T23:31:02.991 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41001.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38997.log 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43433.log 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41371.log.gz 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38997.log.gz 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38625.log 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44600.log 2023-10-07T23:31:02.992 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1%/var/log/ceph/ceph-client.admin.38625.log: -- replaced with /var/log/ceph/ceph-client.admin.43433.log.gz 2023-10-07T23:31:02.993 INFO:teuthology.orchestra.run.smithi057.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz 2023-10-07T23:31:02.993 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2023-10-07T23:31:02.993 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44600.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44600.log.gz 2023-10-07T23:31:02.993 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2023-10-07T23:31:02.993 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2023-10-07T23:31:02.994 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-07T23:31:02.998 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41655.log 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 /var/log/ceph/ceph-client.admin.41655.log: --verbose -- /var/log/ceph/ceph-client.admin.39543.log 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41655.log.gz/var/log/ceph/ceph-client.admin.44036.log: 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33266.log 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2023-10-07T23:31:02.999 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39543.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39543.log.gz 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38785.log 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.33266.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38786.log 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38785.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.33266.log.gz 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38785.log.gz 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38733.log 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38786.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.41926.log.gz 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38786.log.gz 2023-10-07T23:31:03.000 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39196.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38733.log.gz 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38436.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2023-10-07T23:31:03.001 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45196.log 2023-10-07T23:31:03.002 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44117.log 2023-10-07T23:31:03.002 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2023-10-07T23:31:03.002 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.45196.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45196.log.gz/var/log/ceph/ceph-client.admin.44117.log: 2023-10-07T23:31:03.002 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44117.log.gz 2023-10-07T23:31:03.004 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33529.log 2023-10-07T23:31:03.004 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.33529.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33529.log.gz 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.40120.log 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.2.40120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39570.log 2023-10-07T23:31:03.005 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42620.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2023-10-07T23:31:03.006 INFO:teuthology.orchestra.run.smithi057.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.2.40120.log.gz 2023-10-07T23:31:03.006 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2023-10-07T23:31:03.006 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44570.log 2023-10-07T23:31:03.006 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42701.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2023-10-07T23:31:03.006 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39570.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39570.log.gz 2023-10-07T23:31:03.016 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44816.log 2023-10-07T23:31:03.017 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40312.log 2023-10-07T23:31:03.017 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44570.log: /var/log/ceph/ceph-client.admin.44816.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44570.log.gz 2023-10-07T23:31:03.017 INFO:teuthology.orchestra.run.smithi057.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44816.log.gz 2023-10-07T23:31:03.017 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40555.log 2023-10-07T23:31:03.018 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43053.log 2023-10-07T23:31:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40312.log: 88.8%/var/log/ceph/ceph-client.admin.40555.log: -- replaced with /var/log/ceph/ceph-client.admin.40312.log.gz 2023-10-07T23:31:03.019 INFO:teuthology.orchestra.run.smithi057.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2023-10-07T23:31:03.019 INFO:teuthology.orchestra.run.smithi057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40555.log.gz 2023-10-07T23:31:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43053.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43053.log.gz 2023-10-07T23:31:03.019 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42728.log 2023-10-07T23:31:03.022 INFO:teuthology.orchestra.run.smithi057.stderr:gzip /var/log/ceph/ceph-client.admin.44789.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40812.log 2023-10-07T23:31:03.022 INFO:teuthology.orchestra.run.smithi057.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2023-10-07T23:31:03.022 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.42728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42728.log.gz 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40393.log 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40812.log.gz 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44978.log 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40258.log 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40393.log.gz 2023-10-07T23:31:03.023 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.44978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39203.log 2023-10-07T23:31:03.026 INFO:teuthology.orchestra.run.smithi057.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44978.log.gz 2023-10-07T23:31:03.026 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.40258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41545.log 2023-10-07T23:31:03.026 INFO:teuthology.orchestra.run.smithi057.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40258.log.gz 2023-10-07T23:31:03.026 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.39203.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-07T23:31:03.027 INFO:teuthology.orchestra.run.smithi057.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39203.log.gz 2023-10-07T23:31:03.027 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38407.log 2023-10-07T23:31:03.027 INFO:teuthology.orchestra.run.smithi057.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41545.log.gz 2023-10-07T23:31:03.027 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2023-10-07T23:31:03.027 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.38407.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38407.log.gz 2023-10-07T23:31:03.028 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43217.log 2023-10-07T23:31:03.029 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.41817.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2023-10-07T23:31:03.037 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.43217.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43217.log.gz 2023-10-07T23:31:03.095 INFO:teuthology.orchestra.run.smithi154.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-07T23:31:03.102 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-07T23:31:03.109 INFO:teuthology.orchestra.run.smithi057.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-07T23:31:03.438 INFO:teuthology.orchestra.run.smithi057.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-07T23:31:04.431 INFO:teuthology.orchestra.run.smithi057.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-07T23:31:04.873 INFO:teuthology.orchestra.run.smithi154.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-07T23:31:04.930 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-07T23:31:05.131 INFO:teuthology.orchestra.run.smithi057.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-07T23:31:05.644 INFO:teuthology.orchestra.run.smithi154.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-07T23:31:05.690 INFO:teuthology.orchestra.run.smithi154.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-07T23:31:05.940 INFO:teuthology.orchestra.run.smithi057.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-07T23:31:06.112 INFO:teuthology.orchestra.run.smithi057.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-07T23:31:19.094 INFO:teuthology.orchestra.run.smithi057.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.43738.log.gz 2023-10-07T23:31:19.096 INFO:teuthology.orchestra.run.smithi057.stderr: 2023-10-07T23:31:19.096 INFO:teuthology.orchestra.run.smithi057.stderr:real 0m16.245s 2023-10-07T23:31:19.096 INFO:teuthology.orchestra.run.smithi057.stderr:user 0m26.126s 2023-10-07T23:31:19.096 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m0.902s 2023-10-07T23:31:40.063 INFO:teuthology.orchestra.run.smithi154.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-07T23:31:40.064 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-10-07T23:31:40.064 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m37.210s 2023-10-07T23:31:40.064 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m45.863s 2023-10-07T23:31:40.064 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m1.528s 2023-10-07T23:31:40.065 INFO:tasks.ceph:Archiving logs... 2023-10-07T23:31:40.065 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi057/log 2023-10-07T23:31:40.066 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T23:31:44.384 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi154/log 2023-10-07T23:31:44.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-07T23:31:50.761 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-07T23:31:50.772 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-07T23:31:50.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T23:31:50.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-07T23:31:50.844 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-07T23:31:50.855 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-07T23:31:50.855 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:31:50.857 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-07T23:31:50.881 INFO:teuthology.orchestra.run.smithi057.stdout: remote refid st t when poll reach delay offset jitter 2023-10-07T23:31:50.881 INFO:teuthology.orchestra.run.smithi057.stdout:============================================================================== 2023-10-07T23:31:50.881 INFO:teuthology.orchestra.run.smithi057.stdout:*hv01.front.sepi 44.4.53.2 2 u 30 64 377 0.101 0.896 1.550 2023-10-07T23:31:50.882 INFO:teuthology.orchestra.run.smithi057.stdout:+hv02.front.sepi 63.231.80.2 3 u 28 64 377 0.088 0.354 1.590 2023-10-07T23:31:50.882 INFO:teuthology.orchestra.run.smithi057.stdout:+hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.087 0.322 1.568 2023-10-07T23:31:50.882 INFO:teuthology.orchestra.run.smithi057.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2023-10-07T23:31:50.883 INFO:teuthology.orchestra.run.smithi154.stdout: remote refid st t when poll reach delay offset jitter 2023-10-07T23:31:50.883 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================== 2023-10-07T23:31:50.883 INFO:teuthology.orchestra.run.smithi154.stdout:*hv01.front.sepi 44.4.53.2 2 u 30 64 377 0.071 -0.749 0.269 2023-10-07T23:31:50.883 INFO:teuthology.orchestra.run.smithi154.stdout:+hv02.front.sepi 63.231.80.2 3 u 30 64 377 0.102 -1.273 0.267 2023-10-07T23:31:50.884 INFO:teuthology.orchestra.run.smithi154.stdout:+hv03.front.sepi 74.6.168.72 3 u 25 64 377 0.082 -1.660 0.402 2023-10-07T23:31:50.884 INFO:teuthology.orchestra.run.smithi154.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2023-10-07T23:31:50.885 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-07T23:31:50.921 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-07T23:31:50.922 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-07T23:31:50.999 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-07T23:31:51.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-07T23:31:51.064 INFO:teuthology.task.internal:Duration was 1238.036018 seconds 2023-10-07T23:31:51.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-07T23:31:51.098 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-07T23:31:51.098 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T23:31:51.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-07T23:31:51.151 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-07T23:31:51.151 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2023-10-07T23:31:51.151 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T23:31:51.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2023-10-07T23:31:51.162 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-07T23:31:51.173 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-07T23:31:51.174 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T23:31:51.206 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-07T23:31:51.241 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-07T23:31:51.241 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T23:31:51.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-07T23:31:51.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-07T23:31:51.343 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-07T23:31:51.343 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T23:31:51.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-07T23:31:51.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-07T23:31:51.383 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:31:51.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:31:51.414 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2023-10-07T23:31:51.416 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2023-10-07T23:31:51.433 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:31:51.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:31:51.439 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-07T23:31:51.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-07T23:31:51.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-07T23:31:51.550 INFO:teuthology.task.internal:Transferring archived files... 2023-10-07T23:31:51.551 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi057 2023-10-07T23:31:51.551 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T23:31:51.573 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060/remote/smithi154 2023-10-07T23:31:51.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-07T23:31:51.596 INFO:teuthology.task.internal:Removing archive directory... 2023-10-07T23:31:51.596 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T23:31:51.618 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-07T23:31:51.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-07T23:31:51.634 INFO:teuthology.task.internal:Not uploading archives. 2023-10-07T23:31:51.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-07T23:31:51.680 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-07T23:31:51.681 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T23:31:51.683 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-07T23:31:51.688 INFO:teuthology.orchestra.run.smithi057.stdout: 924113 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 23:31 /home/ubuntu/cephtest 2023-10-07T23:31:51.691 INFO:teuthology.orchestra.run.smithi154.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 7 23:31 /home/ubuntu/cephtest 2023-10-07T23:31:51.693 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-07T23:31:51.715 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-07T23:31:51.782 INFO:teuthology.nuke:Checking targets against current locks 2023-10-07T23:31:51.808 DEBUG:teuthology.nuke:shortname: smithi057 2023-10-07T23:31:51.809 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:31:51.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:01:15.903445', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-07T23:31:51.835 DEBUG:teuthology.nuke:shortname: smithi154 2023-10-07T23:31:51.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-07T23:31:51.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-07 23:01:15.901698', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-10-07T23:31:51.891 INFO:teuthology.orchestra.console.smithi057:Power off 2023-10-07T23:31:51.891 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T23:31:51.915 DEBUG:teuthology.orchestra.console.smithi057:power off output: Chassis Power Control: Down/Off 2023-10-07T23:31:51.915 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:31:51.930 INFO:teuthology.orchestra.console.smithi154:Power off 2023-10-07T23:31:51.931 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-07T23:31:51.940 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-07T23:31:51.953 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-10-07T23:31:51.953 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:31:51.979 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-07T23:31:55.941 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:31:55.981 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:31:56.051 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-07T23:31:56.091 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-10-07T23:32:00.052 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:32:00.092 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:32:00.163 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is on 2023-10-07T23:32:00.202 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-10-07T23:32:00.302 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-10-07T23:32:04.166 DEBUG:teuthology.orchestra.console.smithi057:pexpect command: ipmitool -H smithi057.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-07T23:32:04.278 DEBUG:teuthology.orchestra.console.smithi057:check power output: Chassis Power is off 2023-10-07T23:32:04.379 INFO:teuthology.orchestra.console.smithi057:Power off completed 2023-10-07T23:32:04.484 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} duration: 1238.036018371582 failure_reason: 'Test failure: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=794d5b6462a14cc2b83f29ceac4a104f status: fail success: false 2023-10-07T23:32:04.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-07T23:32:04.566 INFO:teuthology.run:FAIL